BV Prod Test Dashboard

BV Mobile (iOS & Android)

iOS Smoke: HEALTHY - ✅ 3 | ❌ 0 - Run Time: 3/12/26 11:59:29 PM EST — Build: 2.2.4View RunTrigger Workflowℹ️
Select environment and set Maestro test tag to 'smoke'
Test Cases
- PASSED - AI Streaming - Verify AI summaries card loads when asking a question and follow-up question, and navigating to the next card shows a standard answer card and PDF loads correctly
- PASSED - Data Validation/Answer Relevance - Verify officer searches "Chapter 266, Section 30", sees matching answer cards, and opens the referenced PDF
- PASSED - File Search - Verify user can expand/collapse file categories, search for a file, open PDF, and switching to another unit reveals new unit-specific files/categories
Previous Runs
- HEALTHY ✅ Pass: 3 | ❌ Fail: 0 - Run Time: 3/12/26 9:20:29 AM EST — Build: 2.2.4View Run
  Test Cases
  - PASSED - File Search - Verify user can expand/collapse file categories, search for a file, open PDF, and switching to another unit reveals new unit-specific files/categories
  - PASSED - AI Streaming - Verify AI summaries card loads when asking a question and follow-up question, and navigating to the next card shows a standard answer card and PDF loads correctly
  - PASSED - Data Validation/Answer Relevance - Verify officer searches "Chapter 266, Section 30", sees matching answer cards, and opens the referenced PDF
- HEALTHY ✅ Pass: 3 | ❌ Fail: 0 - Run Time: 3/10/26 9:22:30 PM EST — Build: 2.2.10View Run
  Test Cases
  - PASSED - File Search - Verify user can expand/collapse file categories, search for a file, open PDF, and switching to another unit reveals new unit-specific files/categories
  - PASSED - AI Streaming - Verify AI summaries card loads when asking a question and follow-up question, and navigating to the next card shows a standard answer card and PDF loads correctly
  - PASSED - Data Validation/Answer Relevance - Verify officer searches "Chapter 266, Section 30", sees matching answer cards, and opens the referenced PDF
- HEALTHY ✅ Pass: 3 | ❌ Fail: 0 - Run Time: 3/9/26 1:57:53 PM ESTView Run
  Test Cases
  - PASSED - Data Validation/Answer Relevance - Verify officer searches "Chapter 266, Section 30", sees matching answer cards, and opens the referenced PDF
  - PASSED - File Search - Verify user can expand/collapse file categories, search for a file, open PDF, and switching to another unit reveals new unit-specific files/categories
  - PASSED - AI Streaming - Verify AI summaries card loads when asking a question and follow-up question, and navigating to the next card shows a standard answer card and PDF loads correctly
iOS Regression: HEALTHY - ✅ 18 | ❌ 0 - Run Time: 3/14/26 12:09:02 AM EST — Build: 2.2.10View RunTrigger Workflowℹ️
Select environment and set Maestro test tag to 'regression'
Test Cases
- PASSED - Fillable PDF through chat - Verify officer can open a fillable PDF from an answer card, enable/disable AI form, draw/undo annotation, fill out form fields, email form to self and additional recipient, and verify email validations
- PASSED - Ask Blue - Search and Navigation - Verify asking a question, viewing results, navigating carousel, selecting/deselecting categories, and asking follow-up question
- PASSED - Tutorials - Verify user can access Tutorials menu and navigate to training video
- PASSED - Never Logged In - Verify never logged in before flow with email and text support options
- PASSED - First-time User/EULA - Verify first-time user flow with EULA/PP agreement
- PASSED - Login Validation - Verify login error messages for various invalid input scenarios
- PASSED - Forgot Password - Verify forgot password flow with validations
- PASSED - Settings - Verify settings functionality including viewing user details, submitting feedback, switching themes and reset password functionality (no full reset password)
- PASSED - Switch Unit - Verify switch unit functionality with and without passcode requirements, both in Ask Blue and Settings
- PASSED - AI Streaming Disabled - Verify AI summaries do not appear when feature is disabled and standard answer cards are shown both in new and follow-up questions
- PASSED - Protected PDF - Verify officer can open a protected PDF document from an answer card, navigate through pages, and return to Ask Blue
- PASSED - History - Verify question history displays recent questions and follow-up questions with correct timestamps, and can reopen chats from history
- PASSED - Ask Blue - Error validations - Verify error validations when asking a question without input, too short input, and asking question and follow-up question in empty category
- PASSED - Data Validation/Answer Relevance - Verify officer asks "Elements of larceny" and sees matching answer cards
- PASSED - Request Account - Verify request account flow with validations
- PASSED - Notifications - Verify user can view, delete individual, and delete all notifications
- PASSED - Sign Off Policy Task - Verify officer can sign off a policy task only after reviewing document first and can review document again
- PASSED - Fillable PDF from Files - Verify officer can open a fillable PDF form the Files tab, enable/disable AI form, draw/undo annotation, fill out form fields, email form to self and additional recipient, and verify email validations
Previous Runs
- HEALTHY ✅ Pass: 18 | ❌ Fail: 0 - Run Time: 3/11/26 12:37:34 AM EST — Build: 2.2.4View Run
  Test Cases
  - PASSED - Fillable PDF from Files - Verify officer can open a fillable PDF form the Files tab, enable/disable AI form, draw/undo annotation, fill out form fields, email form to self and additional recipient, and verify email validations
  - PASSED - Protected PDF - Verify officer can open a protected PDF document from an answer card, navigate through pages, and return to Ask Blue
  - PASSED - Fillable PDF through chat - Verify officer can open a fillable PDF from an answer card, enable/disable AI form, draw/undo annotation, fill out form fields, email form to self and additional recipient, and verify email validations
  - PASSED - History - Verify question history displays recent questions and follow-up questions with correct timestamps, and can reopen chats from history
  - PASSED - Ask Blue - Error validations - Verify error validations when asking a question without input, too short input, and asking question and follow-up question in empty category
  - PASSED - Ask Blue - Search and Navigation - Verify asking a question, viewing results, navigating carousel, selecting/deselecting categories, and asking follow-up question
  - PASSED - Settings - Verify settings functionality including viewing user details, submitting feedback, switching themes and reset password functionality (no full reset password)
  - PASSED - First-time User/EULA - Verify first-time user flow with EULA/PP agreement
  - PASSED - Request Account - Verify request account flow with validations
  - PASSED - Never Logged In - Verify never logged in before flow with email and text support options
  - PASSED - Login Validation - Verify login error messages for various invalid input scenarios
  - PASSED - Forgot Password - Verify forgot password flow with validations
  - PASSED - Sign Off Policy Task - Verify officer can sign off a policy task only after reviewing document first and can review document again
  - PASSED - Tutorials - Verify user can access Tutorials menu and navigate to training video
  - PASSED - AI Streaming Disabled - Verify AI summaries do not appear when feature is disabled and standard answer cards are shown both in new and follow-up questions
  - PASSED - Switch Unit - Verify switch unit functionality with and without passcode requirements, both in Ask Blue and Settings
  - PASSED - Data Validation/Answer Relevance - Verify officer asks "Elements of larceny" and sees matching answer cards
  - PASSED - Notifications - Verify user can view, delete individual, and delete all notifications
- UNSTABLE ✅ Pass: 17 | ❌ Fail: 1 - Run Time: 3/10/26 10:19:09 PM EST — Build: 2.2.10View Run
  Test Cases
  - FAILED - Settings - Verify settings functionality including viewing user details, submitting feedback, switching themes and reset password functionality (no full reset password)
  - PASSED - AI Streaming Disabled - Verify AI summaries do not appear when feature is disabled and standard answer cards are shown both in new and follow-up questions
  - PASSED - Fillable PDF through chat - Verify officer can open a fillable PDF from an answer card, enable/disable AI form, draw/undo annotation, fill out form fields, email form to self and additional recipient, and verify email validations
  - PASSED - History - Verify question history displays recent questions and follow-up questions with correct timestamps, and can reopen chats from history
  - PASSED - Ask Blue - Error validations - Verify error validations when asking a question without input, too short input, and asking question and follow-up question in empty category
  - PASSED - Ask Blue - Search and Navigation - Verify asking a question, viewing results, navigating carousel, selecting/deselecting categories, and asking follow-up question
  - PASSED - Tutorials - Verify user can access Tutorials menu and navigate to training video
  - PASSED - Data Validation/Answer Relevance - Verify officer asks "Elements of larceny" and sees matching answer cards
  - PASSED - Request Account - Verify request account flow with validations
  - PASSED - Never Logged In - Verify never logged in before flow with email and text support options
  - PASSED - Login Validation - Verify login error messages for various invalid input scenarios
  - PASSED - Forgot Password - Verify forgot password flow with validations
  - PASSED - Notifications - Verify user can view, delete individual, and delete all notifications
  - PASSED - Sign Off Policy Task - Verify officer can sign off a policy task only after reviewing document first and can review document again
  - PASSED - Fillable PDF from Files - Verify officer can open a fillable PDF form the Files tab, enable/disable AI form, draw/undo annotation, fill out form fields, email form to self and additional recipient, and verify email validations
  - PASSED - Switch Unit - Verify switch unit functionality with and without passcode requirements, both in Ask Blue and Settings
  - PASSED - Protected PDF - Verify officer can open a protected PDF document from an answer card, navigate through pages, and return to Ask Blue
  - PASSED - First-time User/EULA - Verify first-time user flow with EULA/PP agreement
- UNSTABLE ✅ Pass: 17 | ❌ Fail: 1 - Run Time: 3/9/26 1:29:43 PM EST — Build: 2.2.9View Run
  Test Cases
  - FAILED - AI Streaming Disabled - Verify AI summaries do not appear when feature is disabled and standard answer cards are shown both in new and follow-up questions
  - PASSED - Settings - Verify settings functionality including viewing user details, submitting feedback, switching themes and reset password functionality (no full reset password)
  - PASSED - Fillable PDF from Files - Verify officer can open a fillable PDF form the Files tab, enable/disable AI form, draw/undo annotation, fill out form fields, email form to self and additional recipient, and verify email validations
  - PASSED - Switch Unit - Verify switch unit functionality with and without passcode requirements, both in Ask Blue and Settings
  - PASSED - Protected PDF - Verify officer can open a protected PDF document from an answer card, navigate through pages, and return to Ask Blue
  - PASSED - Fillable PDF through chat - Verify officer can open a fillable PDF from an answer card, enable/disable AI form, draw/undo annotation, fill out form fields, email form to self and additional recipient, and verify email validations
  - PASSED - History - Verify question history displays recent questions and follow-up questions with correct timestamps, and can reopen chats from history
  - PASSED - Tutorials - Verify user can access Tutorials menu and navigate to training video
  - PASSED - Ask Blue - Error validations - Verify error validations when asking a question without input, too short input, and asking question and follow-up question in empty category
  - PASSED - Ask Blue - Search and Navigation - Verify asking a question, viewing results, navigating carousel, selecting/deselecting categories, and asking follow-up question
  - PASSED - First-time User/EULA - Verify first-time user flow with EULA/PP agreement
  - PASSED - Data Validation/Answer Relevance - Verify officer asks "Elements of larceny" and sees matching answer cards
  - PASSED - Request Account - Verify request account flow with validations
  - PASSED - Never Logged In - Verify never logged in before flow with email and text support options
  - PASSED - Login Validation - Verify login error messages for various invalid input scenarios
  - PASSED - Forgot Password - Verify forgot password flow with validations
  - PASSED - Notifications - Verify user can view, delete individual, and delete all notifications
  - PASSED - Sign Off Policy Task - Verify officer can sign off a policy task only after reviewing document first and can review document again
Android Smoke: HEALTHY - ✅ 3 | ❌ 0 - Run Time: 3/12/26 11:59:42 PM EST — Build: 2.2.4View RunTrigger Workflowℹ️
Select environment and set Maestro test tag to 'smoke'
Test Cases
- PASSED - File Search - Verify user can expand/collapse file categories, search for a file, open PDF, and switching to another unit reveals new unit-specific files/categories
- PASSED - AI Streaming - Verify AI summaries card loads when asking a question and follow-up question, and navigating to the next card shows a standard answer card and PDF loads correctly
- PASSED - Data Validation/Answer Relevance - Verify officer searches "Chapter 266, Section 30", sees matching answer cards, and opens the referenced PDF
Previous Runs
- HEALTHY ✅ Pass: 3 | ❌ Fail: 0 - Run Time: 3/12/26 9:21:14 AM EST — Build: 2.2.4View Run
  Test Cases
  - PASSED - AI Streaming - Verify AI summaries card loads when asking a question and follow-up question, and navigating to the next card shows a standard answer card and PDF loads correctly
  - PASSED - File Search - Verify user can expand/collapse file categories, search for a file, open PDF, and switching to another unit reveals new unit-specific files/categories
  - PASSED - Data Validation/Answer Relevance - Verify officer searches "Chapter 266, Section 30", sees matching answer cards, and opens the referenced PDF
- HEALTHY ✅ Pass: 3 | ❌ Fail: 0 - Run Time: 3/11/26 9:37:29 AM EST — Build: 2.2.4View Run
  Test Cases
  - PASSED - AI Streaming - Verify AI summaries card loads when asking a question and follow-up question, and navigating to the next card shows a standard answer card and PDF loads correctly
  - PASSED - Data Validation/Answer Relevance - Verify officer searches "Chapter 266, Section 30", sees matching answer cards, and opens the referenced PDF
  - PASSED - File Search - Verify user can expand/collapse file categories, search for a file, open PDF, and switching to another unit reveals new unit-specific files/categories
- HEALTHY ✅ Pass: 3 | ❌ Fail: 0 - Run Time: 3/10/26 9:20:58 AM EST — Build: 2.2.4View Run
  Test Cases
  - PASSED - File Search - Verify user can expand/collapse file categories, search for a file, open PDF, and switching to another unit reveals new unit-specific files/categories
  - PASSED - AI Streaming - Verify AI summaries card loads when asking a question and follow-up question, and navigating to the next card shows a standard answer card and PDF loads correctly
  - PASSED - Data Validation/Answer Relevance - Verify officer searches "Chapter 266, Section 30", sees matching answer cards, and opens the referenced PDF
Android Regression: HEALTHY - ✅ 18 | ❌ 0 - Run Time: 3/14/26 12:09:08 AM EST — Build: 2.2.10View RunTrigger Workflowℹ️
Select environment and set Maestro test tag to 'regression'
Test Cases
- PASSED - Protected PDF - Verify officer can open a protected PDF document from an answer card, navigate through pages, and return to Ask Blue
- PASSED - Tutorials - Verify user can access Tutorials menu and navigate to training video
- PASSED - Request Account - Verify request account flow with validations
- PASSED - Login Validation - Verify login error messages for various invalid input scenarios
- PASSED - First-time User/EULA - Verify first-time user flow with EULA/PP agreement
- PASSED - Notifications - Verify user can view, delete individual, and delete all notifications
- PASSED - Sign Off Policy Task - Verify officer can sign off a policy task only after reviewing document first and can review document again
- PASSED - Settings - Verify settings functionality including viewing user details, submitting feedback, switching themes and reset password functionality (no full reset password)
- PASSED - Switch Unit - Verify switch unit functionality with and without passcode requirements, both in Ask Blue and Settings
- PASSED - Fillable PDF through chat - Verify officer can open a fillable PDF from an answer card, enable/disable AI form, draw/undo annotation, fill out form fields, email form to self and additional recipient, and verify email validations
- PASSED - History - Verify question history displays recent questions and follow-up questions with correct timestamps, and can reopen chats from history
- PASSED - Ask Blue - Error validations - Verify error validations when asking a question without input, too short input, and asking question and follow-up question in empty category
- PASSED - Ask Blue - Search and Navigation - Verify asking a question, viewing results, navigating carousel, selecting/deselecting categories, and asking follow-up question
- PASSED - Data Validation/Answer Relevance - Verify officer asks "Elements of larceny" and sees matching answer cards
- PASSED - Never Logged In - Verify never logged in before flow with email and text support options
- PASSED - Forgot Password - Verify forgot password flow with validations
- PASSED - Fillable PDF from Files - Verify officer can open a fillable PDF form the Files tab, enable/disable AI form, draw/undo annotation, fill out form fields, email form to self and additional recipient, and verify email validations
- PASSED - AI Streaming Disabled - Verify AI summaries do not appear when feature is disabled and standard answer cards are shown both in new and follow-up questions
Previous Runs
- HEALTHY ✅ Pass: 18 | ❌ Fail: 0 - Run Time: 3/10/26 12:27:44 PM EST — Build: 2.2.10View Run
  Test Cases
  - PASSED - Settings - Verify settings functionality including viewing user details, submitting feedback, switching themes and reset password functionality (no full reset password)
  - PASSED - Protected PDF - Verify officer can open a protected PDF document from an answer card, navigate through pages, and return to Ask Blue
  - PASSED - Fillable PDF through chat - Verify officer can open a fillable PDF from an answer card, enable/disable AI form, draw/undo annotation, fill out form fields, email form to self and additional recipient, and verify email validations
  - PASSED - History - Verify question history displays recent questions and follow-up questions with correct timestamps, and can reopen chats from history
  - PASSED - Ask Blue - Search and Navigation - Verify asking a question, viewing results, navigating carousel, selecting/deselecting categories, and asking follow-up question
  - PASSED - Fillable PDF from Files - Verify officer can open a fillable PDF form the Files tab, enable/disable AI form, draw/undo annotation, fill out form fields, email form to self and additional recipient, and verify email validations
  - PASSED - Switch Unit - Verify switch unit functionality with and without passcode requirements, both in Ask Blue and Settings
  - PASSED - AI Streaming Disabled - Verify AI summaries do not appear when feature is disabled and standard answer cards are shown both in new and follow-up questions
  - PASSED - Ask Blue - Error validations - Verify error validations when asking a question without input, too short input, and asking question and follow-up question in empty category
  - PASSED - Request Account - Verify request account flow with validations
  - PASSED - First-time User/EULA - Verify first-time user flow with EULA/PP agreement
  - PASSED - Tutorials - Verify user can access Tutorials menu and navigate to training video
  - PASSED - Data Validation/Answer Relevance - Verify officer asks "Elements of larceny" and sees matching answer cards
  - PASSED - Never Logged In - Verify never logged in before flow with email and text support options
  - PASSED - Login Validation - Verify login error messages for various invalid input scenarios
  - PASSED - Forgot Password - Verify forgot password flow with validations
  - PASSED - Notifications - Verify user can view, delete individual, and delete all notifications
  - PASSED - Sign Off Policy Task - Verify officer can sign off a policy task only after reviewing document first and can review document again
- HEALTHY ✅ Pass: 18 | ❌ Fail: 0 - Run Time: 3/5/26 9:03:08 PM EST — Build: 2.2.4View Run
  Test Cases
  - PASSED - Settings - Verify settings functionality including viewing user details, submitting feedback, switching themes and reset password functionality (no full reset password)
  - PASSED - AI Streaming Disabled - Verify AI summaries do not appear when feature is disabled and standard answer cards are shown both in new and follow-up questions
  - PASSED - Protected PDF - Verify officer can open a protected PDF document from an answer card, navigate through pages, and return to Ask Blue
  - PASSED - Ask Blue - Error validations - Verify error validations when asking a question without input, too short input, and asking question and follow-up question in empty category
  - PASSED - Ask Blue - Search and Navigation - Verify asking a question, viewing results, navigating carousel, selecting/deselecting categories, and asking follow-up question
  - PASSED - Tutorials - Verify user can access Tutorials menu and navigate to training video
  - PASSED - First-time User/EULA - Verify first-time user flow with EULA/PP agreement
  - PASSED - Login Validation - Verify login error messages for various invalid input scenarios
  - PASSED - Forgot Password - Verify forgot password flow with validations
  - PASSED - Notifications - Verify user can view, delete individual, and delete all notifications
  - PASSED - Sign Off Policy Task - Verify officer can sign off a policy task only after reviewing document first and can review document again
  - PASSED - Fillable PDF from Files - Verify officer can open a fillable PDF form the Files tab, enable/disable AI form, draw/undo annotation, fill out form fields, email form to self and additional recipient, and verify email validations
  - PASSED - Switch Unit - Verify switch unit functionality with and without passcode requirements, both in Ask Blue and Settings
  - PASSED - Fillable PDF through chat - Verify officer can open a fillable PDF from an answer card, enable/disable AI form, draw/undo annotation, fill out form fields, email form to self and additional recipient, and verify email validations
  - PASSED - History - Verify question history displays recent questions and follow-up questions with correct timestamps, and can reopen chats from history
  - PASSED - Data Validation/Answer Relevance - Verify officer asks "Elements of larceny" and sees matching answer cards
  - PASSED - Request Account - Verify request account flow with validations
  - PASSED - Never Logged In - Verify never logged in before flow with email and text support options
- HEALTHY ✅ Pass: 18 | ❌ Fail: 0 - Run Time: 3/3/26 10:34:44 AM EST — Build: 2.2.4View Run
  Test Cases
  - PASSED - Settings - Verify settings functionality including viewing user details, submitting feedback, switching themes and reset password functionality (no full reset password)
  - PASSED - Fillable PDF from Files - Verify officer can open a fillable PDF form the Files tab, enable/disable AI form, draw/undo annotation, fill out form fields, email form to self and additional recipient, and verify email validations
  - PASSED - Switch Unit - Verify switch unit functionality with and without passcode requirements, both in Ask Blue and Settings
  - PASSED - AI Streaming Disabled - Verify AI summaries do not appear when feature is disabled and standard answer cards are shown both in new and follow-up questions
  - PASSED - Protected PDF - Verify officer can open a protected PDF document from an answer card, navigate through pages, and return to Ask Blue
  - PASSED - Fillable PDF through chat - Verify officer can open a fillable PDF from an answer card, enable/disable AI form, draw/undo annotation, fill out form fields, email form to self and additional recipient, and verify email validations
  - PASSED - History - Verify question history displays recent questions and follow-up questions with correct timestamps, and can reopen chats from history
  - PASSED - Ask Blue - Error validations - Verify error validations when asking a question without input, too short input, and asking question and follow-up question in empty category
  - PASSED - Tutorials - Verify user can access Tutorials menu and navigate to training video
  - PASSED - Ask Blue - Search and Navigation - Verify asking a question, viewing results, navigating carousel, selecting/deselecting categories, and asking follow-up question
  - PASSED - First-time User/EULA - Verify first-time user flow with EULA/PP agreement
  - PASSED - Data Validation/Answer Relevance - Verify officer asks "Elements of larceny" and sees matching answer cards
  - PASSED - Request Account - Verify request account flow with validations
  - PASSED - Never Logged In - Verify never logged in before flow with email and text support options
  - PASSED - Login Validation - Verify login error messages for various invalid input scenarios
  - PASSED - Forgot Password - Verify forgot password flow with validations
  - PASSED - Notifications - Verify user can view, delete individual, and delete all notifications
  - PASSED - Sign Off Policy Task - Verify officer can sign off a policy task only after reviewing document first and can review document again

RV Mobile (iOS & Android)

iOS Smoke: HEALTHY - ✅ Pass: 3 | ❌ Fail: 0 - Run Time: 3/10/26 3:25:56 PM ESTView RunTrigger Workflow
Test Cases
- PASSED - AI Streaming - Verify AI summaries card loads when asking a question and follow-up question, and navigating to the next card shows a standard answer card and PDF loads correctly
- PASSED - File Search - Verify user can expand/collapse file categories, search for a file, open PDF, and switching to another unit reveals new unit-specific files/categories
- PASSED - Data Validation/Answer Relevance - Verify officer searches "Chapter 266, Section 30", sees matching answer cards, and opens the referenced PDF
Previous Runs
- UNSTABLE ✅ Pass: 2 | ❌ Fail: 1 - Run Time: 3/10/26 1:30:47 PM ESTView Run
  Test Cases
  - FAILED - File Search - Verify user can expand/collapse file categories, search for a file, open PDF, and switching to another unit reveals new unit-specific files/categories
  - PASSED - AI Streaming - Verify AI summaries card loads when asking a question and follow-up question, and navigating to the next card shows a standard answer card and PDF loads correctly
  - PASSED - Data Validation/Answer Relevance - Verify officer searches "Chapter 266, Section 30", sees matching answer cards, and opens the referenced PDF
- HEALTHY ✅ Pass: 3 | ❌ Fail: 0 - Run Time: 3/9/26 2:27:32 PM ESTView Run
  Test Cases
  - PASSED - File Search - Verify user can expand/collapse file categories, search for a file, open PDF, and switching to another unit reveals new unit-specific files/categories
  - PASSED - AI Streaming - Verify AI summaries card loads when asking a question and follow-up question, and navigating to the next card shows a standard answer card and PDF loads correctly
  - PASSED - Data Validation/Answer Relevance - Verify officer searches "Chapter 266, Section 30", sees matching answer cards, and opens the referenced PDF
iOS Regression: HEALTHY - ✅ Pass: 18 | ❌ Fail: 0 - Run Time: 3/10/26 2:13:43 PM ESTView RunTrigger Workflow
Test Cases
- PASSED - Fillable PDF from Files - Verify officer can open a fillable PDF form the Files tab, enable/disable AI form, draw/undo annotation, fill out form fields, email form to self and additional recipient, and verify email validations
- PASSED - Settings - Verify settings functionality including viewing user details, submitting feedback, switching themes and reset password functionality (no full reset password)
- PASSED - AI Streaming Disabled - Verify AI summaries do not appear when feature is disabled and standard answer cards are shown both in new and follow-up questions
- PASSED - Switch Unit - Verify switch unit functionality with and without passcode requirements, both in Ask Blue and Settings
- PASSED - Fillable PDF through chat - Verify officer can open a fillable PDF from an answer card, enable/disable AI form, draw/undo annotation, fill out form fields, email form to self and additional recipient, and verify email validations
- PASSED - Protected PDF - Verify officer can open a protected PDF document from an answer card, navigate through pages, and return to Ask Blue
- PASSED - History - Verify question history displays recent questions and follow-up questions with correct timestamps, and can reopen chats from history
- PASSED - Ask Blue - Error validations - Verify error validations when asking a question without input, too short input, and asking question and follow-up question in empty category
- PASSED - Ask Blue - Search and Navigation - Verify asking a question, viewing results, navigating carousel, selecting/deselecting categories, and asking follow-up question
- PASSED - Tutorials - Verify user can access Tutorials menu and navigate to training video
- PASSED - First-time User/EULA - Verify first-time user flow with EULA/PP agreement
- PASSED - Data Validation/Answer Relevance - Verify officer asks "Elements of larceny" and sees matching answer cards
- PASSED - Request Account - Verify request account flow with validations
- PASSED - Never Logged In - Verify never logged in before flow with email and text support options
- PASSED - Login Validation - Verify login error messages for various invalid input scenarios
- PASSED - Forgot Password - Verify forgot password flow with validations
- PASSED - Notifications - Verify user can view, delete individual, and delete all notifications
- PASSED - Sign Off Policy Task - Verify officer can sign off a policy task only after reviewing document first and can review document again
Android Smoke: HEALTHY - ✅ Pass: 3 | ❌ Fail: 0 - Run Time: 3/9/26 2:27:52 PM ESTView RunTrigger Workflow
Test Cases
- PASSED - File Search - Verify user can expand/collapse file categories, search for a file, open PDF, and switching to another unit reveals new unit-specific files/categories
- PASSED - AI Streaming - Verify AI summaries card loads when asking a question and follow-up question, and navigating to the next card shows a standard answer card and PDF loads correctly
- PASSED - Data Validation/Answer Relevance - Verify officer searches "Chapter 266, Section 30", sees matching answer cards, and opens the referenced PDF
Android Regression: HEALTHY - ✅ Pass: 18 | ❌ Fail: 0 - Run Time: 3/10/26 2:13:31 PM ESTView RunTrigger Workflow
Test Cases
- PASSED - Settings - Verify settings functionality including viewing user details, submitting feedback, switching themes and reset password functionality (no full reset password)
- PASSED - Fillable PDF from Files - Verify officer can open a fillable PDF form the Files tab, enable/disable AI form, draw/undo annotation, fill out form fields, email form to self and additional recipient, and verify email validations
- PASSED - Switch Unit - Verify switch unit functionality with and without passcode requirements, both in Ask Blue and Settings
- PASSED - AI Streaming Disabled - Verify AI summaries do not appear when feature is disabled and standard answer cards are shown both in new and follow-up questions
- PASSED - Protected PDF - Verify officer can open a protected PDF document from an answer card, navigate through pages, and return to Ask Blue
- PASSED - Fillable PDF through chat - Verify officer can open a fillable PDF from an answer card, enable/disable AI form, draw/undo annotation, fill out form fields, email form to self and additional recipient, and verify email validations
- PASSED - Ask Blue - Error validations - Verify error validations when asking a question without input, too short input, and asking question and follow-up question in empty category
- PASSED - History - Verify question history displays recent questions and follow-up questions with correct timestamps, and can reopen chats from history
- PASSED - Ask Blue - Search and Navigation - Verify asking a question, viewing results, navigating carousel, selecting/deselecting categories, and asking follow-up question
- PASSED - Tutorials - Verify user can access Tutorials menu and navigate to training video
- PASSED - Data Validation/Answer Relevance - Verify officer asks "Elements of larceny" and sees matching answer cards
- PASSED - First-time User/EULA - Verify first-time user flow with EULA/PP agreement
- PASSED - Request Account - Verify request account flow with validations
- PASSED - Never Logged In - Verify never logged in before flow with email and text support options
- PASSED - Login Validation - Verify login error messages for various invalid input scenarios
- PASSED - Forgot Password - Verify forgot password flow with validations
- PASSED - Sign Off Policy Task - Verify officer can sign off a policy task only after reviewing document first and can review document again
- PASSED - Notifications - Verify user can view, delete individual, and delete all notifications

Web (Playwright)

Smoke (Cross-state): HEALTHY - ✅ Pass: 3 | ❌ Fail: 0 | 🟡 Flaky: 0 | ⏩ Skipped: 0 - Run Time: 3/5/26 10:08:27 PM ESTView RunTrigger Workflowℹ️
Select environment and optionally enter one or more states (e.g. Texas, New York). Leave blank to run all. Valid states: Massachusetts, California, Connecticut, Texas, New York, Florida.
Test Cases
- PASSED - Massachusetts officer asks "Elements of theft" and opens answer PDF
- PASSED - Massachusetts officer asks "Chapter 266, Section 30" and opens answer PDF
- PASSED - Massachusetts officer asks "Show me the code for Chapter 266, Section 30" and opens answer PDF
Previous Runs
- UNSTABLE ✅ Pass: 15 | ❌ Fail: 3 | 🟡 Flaky: 0 | ⏩ Skipped: 0 - Run Time: 3/5/26 7:33:50 PM ESTView RunView HTML Report
  Test Cases
  - PASSED - Massachusetts officer asks "Elements of theft" and opens answer PDF
  - PASSED - Massachusetts officer asks "Chapter 266, Section 30" and opens answer PDF
  - PASSED - Massachusetts officer asks "Show me the code for Chapter 266, Section 30" and opens answer PDF
  - PASSED - California officer asks "Elements of theft" and opens answer PDF
  - PASSED - California officer asks "PC 484" and opens answer PDF
  - PASSED - California officer asks "Show me the Penal Code for 484" and opens answer PDF
  - PASSED - Connecticut officer asks "Elements of theft" and opens answer PDF
  - PASSED - Connecticut officer asks "53a-119" and opens answer PDF
  - PASSED - Connecticut officer asks "Show me the code for Section 53a-119" and opens answer PDF
  - PASSED - Florida officer asks "Elements of theft" and opens answer PDF
  - PASSED - Florida officer asks "812.014" and opens answer PDF
  - PASSED - Florida officer asks "Show me the code for Section 812.014" and opens answer PDF
  - PASSED - Texas officer asks "Elements of theft" and opens answer PDF
  - FAILED - Texas officer asks "31.03" and opens answer PDF
  - FAILED - Texas officer asks "Show me the Penal Code for Section 31.03" and opens answer PDF
  - PASSED - New York officer asks "Elements of theft" and opens answer PDF
  - PASSED - New York officer asks "155.25" and opens answer PDF
  - FAILED - New York officer asks "Show me the code for 155.25" and opens answer PDF
- HEALTHY ✅ Pass: 3 | ❌ Fail: 0 | 🟡 Flaky: 0 | ⏩ Skipped: 0 - Run Time: 3/4/26 5:44:11 PM ESTView RunView HTML Report
  Test Cases
  - PASSED - Connecticut officer asks "Elements of theft" and opens answer PDF
  - PASSED - Connecticut officer asks "53a-119" and opens answer PDF
  - PASSED - Connecticut officer asks "Show me the code for Section 53a-119" and opens answer PDF
Regression (User): HEALTHY - ✅ Pass: 40 | ❌ Fail: 0 | 🟡 Flaky: 0 | ⏩ Skipped: 0 - Run Time: 3/10/26 10:39:18 AM ESTView RunTrigger Workflowℹ️
Select environment and run Web E2E tests for user flows (test_scope=user).
Test Cases
- PASSED - authenticate as admin
- PASSED - authenticate as user
- PASSED - authenticate as streaming user
- PASSED - @user opens default PDF from Files and sees viewer controls + search
- PASSED - @user opens fillable PDF from Files, uses AI Form, fills a field, and sends email
- PASSED - @user opens default PDF from Chat and uses viewer controls + search
- PASSED - @user opens fillable PDF from Chat, uses AI Form, fills a field, and sends email
- PASSED - @user Open Protected PDF from Chat: allowed controls only, bounded page window (±5)
- PASSED - @user sees no results when searching for a non-existent file
- PASSED - @user searches for a file and opens its PDF from the Files tab
- PASSED - @user switches unit, returns to Files, searches by keyword, and opens a file
- PASSED - @user completes first-time login, accepts EULA & Privacy Policy, and reaches chat
- PASSED - @user sees validation errors for empty email and password
- PASSED - @user shows validation error when email is empty
- PASSED - @user shows validation error when password is empty
- PASSED - @user shows validation error for invalid email format
- PASSED - @user sees error when logging in with non-existent account
- PASSED - @user sees login error when password is incorrect
- PASSED - @user toggles password visibility (show/hide)
- PASSED - @user is redirected to login when accessing protected routes while unauthenticated
- PASSED - @user toggles categories, submits question, pages results carousel, and opens answer PDF
- PASSED - @user switches to passcode-protected unit, asks question, opens PDF, then returns to Patrol
- PASSED - @user sees “No Results” when selected category has no matching documents
- PASSED - @user sees new question in history and reloads its answers from history card
- PASSED - @user asks follow-up in active chat and filters results with category chips
- PASSED - @user reloads existing chat from history and sees follow-up question preserved
- PASSED - @user submits request account form successfully with valid details
- PASSED - @user sees email validation error when email field is empty
- PASSED - @user sees email validation error for invalid email format
- PASSED - @user navigates back to Login page using “Sign in” link
- PASSED - @user closes Request Account form and returns to Login page
- PASSED - @user views profile info
- PASSED - @user toggles between light and dark mode
- PASSED - @user submits feedback
- PASSED - @user can request a reset code to change password and sees failure when using dummy reset code
- PASSED - @user reviews and signs off multi-doc
- PASSED - @user sees AI summary cards when AI Summaries is enabled
- PASSED - @user sees only regular answer cards when AI Summaries is disabled
- PASSED - @user creates workspace, adds docs, asks question, returns, deletes workspace
- PASSED - @user creates SENSITIVE workspace, adds docs, asks question, returns, deletes workspace
Previous Runs
- HEALTHY ✅ Pass: 39 | ❌ Fail: 0 | 🟡 Flaky: 1 | ⏩ Skipped: 0 - Run Time: 3/9/26 8:13:59 PM ESTView RunView HTML Report
  Test Cases
  - PASSED - authenticate as admin
  - PASSED - authenticate as user
  - PASSED - authenticate as streaming user
  - PASSED - @user opens default PDF from Files and sees viewer controls + search
  - PASSED - @user opens fillable PDF from Files, uses AI Form, fills a field, and sends email
  - PASSED - @user opens default PDF from Chat and uses viewer controls + search
  - PASSED - @user opens fillable PDF from Chat, uses AI Form, fills a field, and sends email
  - PASSED - @user Open Protected PDF from Chat: allowed controls only, bounded page window (±5)
  - PASSED - @user sees no results when searching for a non-existent file
  - PASSED - @user searches for a file and opens its PDF from the Files tab
  - PASSED - @user switches unit, returns to Files, searches by keyword, and opens a file
  - PASSED - @user completes first-time login, accepts EULA & Privacy Policy, and reaches chat
  - PASSED - @user sees validation errors for empty email and password
  - PASSED - @user shows validation error when email is empty
  - PASSED - @user shows validation error when password is empty
  - PASSED - @user shows validation error for invalid email format
  - PASSED - @user sees error when logging in with non-existent account
  - PASSED - @user sees login error when password is incorrect
  - PASSED - @user toggles password visibility (show/hide)
  - PASSED - @user is redirected to login when accessing protected routes while unauthenticated
  - PASSED - @user toggles categories, submits question, pages results carousel, and opens answer PDF
  - PASSED - @user switches to passcode-protected unit, asks question, opens PDF, then returns to Patrol
  - PASSED - @user sees “No Results” when selected category has no matching documents
  - PASSED - @user sees new question in history and reloads its answers from history card
  - PASSED - @user asks follow-up in active chat and filters results with category chips
  - PASSED - @user reloads existing chat from history and sees follow-up question preserved
  - PASSED - @user submits request account form successfully with valid details
  - PASSED - @user sees email validation error when email field is empty
  - PASSED - @user sees email validation error for invalid email format
  - PASSED - @user navigates back to Login page using “Sign in” link
  - PASSED - @user closes Request Account form and returns to Login page
  - PASSED - @user views profile info
  - PASSED - @user toggles between light and dark mode
  - PASSED - @user submits feedback
  - FLAKY - @user can request a reset code to change password and sees failure when using dummy reset code
  - PASSED - @user reviews and signs off multi-doc
  - PASSED - @user sees AI summary cards when AI Summaries is enabled
  - PASSED - @user sees only regular answer cards when AI Summaries is disabled
  - PASSED - @user creates workspace, adds docs, asks question, returns, deletes workspace
  - PASSED - @user creates SENSITIVE workspace, adds docs, asks question, returns, deletes workspace
- HEALTHY ✅ Pass: 39 | ❌ Fail: 0 | 🟡 Flaky: 1 | ⏩ Skipped: 0 - Run Time: 3/9/26 7:33:04 PM ESTView RunView HTML Report
  Test Cases
  - PASSED - authenticate as admin
  - PASSED - authenticate as user
  - PASSED - authenticate as streaming user
  - PASSED - @user opens default PDF from Files and sees viewer controls + search
  - PASSED - @user opens fillable PDF from Files, uses AI Form, fills a field, and sends email
  - PASSED - @user opens default PDF from Chat and uses viewer controls + search
  - PASSED - @user opens fillable PDF from Chat, uses AI Form, fills a field, and sends email
  - PASSED - @user Open Protected PDF from Chat: allowed controls only, bounded page window (±5)
  - PASSED - @user sees no results when searching for a non-existent file
  - PASSED - @user searches for a file and opens its PDF from the Files tab
  - PASSED - @user switches unit, returns to Files, searches by keyword, and opens a file
  - PASSED - @user completes first-time login, accepts EULA & Privacy Policy, and reaches chat
  - PASSED - @user sees validation errors for empty email and password
  - PASSED - @user shows validation error when email is empty
  - PASSED - @user shows validation error when password is empty
  - PASSED - @user shows validation error for invalid email format
  - PASSED - @user sees error when logging in with non-existent account
  - PASSED - @user sees login error when password is incorrect
  - PASSED - @user toggles password visibility (show/hide)
  - PASSED - @user is redirected to login when accessing protected routes while unauthenticated
  - PASSED - @user toggles categories, submits question, pages results carousel, and opens answer PDF
  - PASSED - @user switches to passcode-protected unit, asks question, opens PDF, then returns to Patrol
  - PASSED - @user sees “No Results” when selected category has no matching documents
  - PASSED - @user sees new question in history and reloads its answers from history card
  - PASSED - @user asks follow-up in active chat and filters results with category chips
  - PASSED - @user reloads existing chat from history and sees follow-up question preserved
  - PASSED - @user submits request account form successfully with valid details
  - PASSED - @user sees email validation error when email field is empty
  - PASSED - @user sees email validation error for invalid email format
  - PASSED - @user navigates back to Login page using “Sign in” link
  - PASSED - @user closes Request Account form and returns to Login page
  - PASSED - @user views profile info
  - PASSED - @user toggles between light and dark mode
  - PASSED - @user submits feedback
  - PASSED - @user can request a reset code to change password and sees failure when using dummy reset code
  - PASSED - @user reviews and signs off multi-doc
  - PASSED - @user sees AI summary cards when AI Summaries is enabled
  - FLAKY - @user sees only regular answer cards when AI Summaries is disabled
  - PASSED - @user creates workspace, adds docs, asks question, returns, deletes workspace
  - PASSED - @user creates SENSITIVE workspace, adds docs, asks question, returns, deletes workspace
Regression (Admin): UNSTABLE - ✅ Pass: 22 | ❌ Fail: 2 | 🟡 Flaky: 0 | ⏩ Skipped: 0 - Run Time: 3/10/26 11:59:01 AM ESTView RunTrigger Workflowℹ️
Select environment and run Web E2E tests for admin flows (test_scope=admin).
Test Cases
- PASSED - authenticate as admin
- PASSED - authenticate as user
- PASSED - authenticate as streaming user
- PASSED - @admin creates, edits, and deletes a category
- PASSED - @admin reorders created categories
- PASSED - @admin uploads a new PDF with categories, verifies it, and deletes it
- PASSED - @admin views existing document and renames it
- PASSED - @admin archives a document, verifies filter behavior, and unarchives it
- PASSED - @admin verifies SuperDoc PDF is hidden when “Show Hidden” filter is applied
- PASSED - @admin sets default email addresses for uploaded doc, verifies persistence, and cleans up
- PASSED - @admin error validation when fields are missing
- PASSED - @admin error validation when email is invalid
- PASSED - @admin error validation when password is incorrect
- PASSED - @admin error validation when user email is not an admin
- PASSED - @admin valid login and logout
- PASSED - @admin toggle password visibility
- PASSED - @admin can request a reset code and sees failure with dummy code
- PASSED - @admin views document from a multi-doc sign off task
- PASSED - @admin edits an existing sign off task (name & description)
- PASSED - @admin downloads a sign off report
- PASSED - @admin archives a task and verifies Download Report and View Document in Archived table
- FAILED - @admin creates, updates (categories + passcode), and deletes a unit
- PASSED - @admin creates, edits, toggles role, and deletes a user
- FAILED - @admin resets a user password and can view dept temp password
Previous Runs
- HEALTHY ✅ Pass: 23 | ❌ Fail: 0 | 🟡 Flaky: 1 | ⏩ Skipped: 0 - Run Time: 3/9/26 10:51:41 AM ESTView RunView HTML Report
  Test Cases
  - PASSED - authenticate as admin
  - PASSED - authenticate as user
  - PASSED - authenticate as streaming user
  - PASSED - @admin creates, edits, and deletes a category
  - PASSED - @admin reorders created categories
  - PASSED - @admin uploads a new PDF with categories, verifies it, and deletes it
  - PASSED - @admin views existing document and renames it
  - PASSED - @admin archives a document, verifies filter behavior, and unarchives it
  - PASSED - @admin verifies SuperDoc PDF is hidden when “Show Hidden” filter is applied
  - PASSED - @admin sets default email addresses for uploaded doc, verifies persistence, and cleans up
  - PASSED - @admin error validation when fields are missing
  - PASSED - @admin error validation when email is invalid
  - PASSED - @admin error validation when password is incorrect
  - PASSED - @admin error validation when user email is not an admin
  - PASSED - @admin valid login and logout
  - PASSED - @admin toggle password visibility
  - PASSED - @admin can request a reset code and sees failure with dummy code
  - PASSED - @admin views document from a multi-doc sign off task
  - PASSED - @admin edits an existing sign off task (name & description)
  - PASSED - @admin downloads a sign off report
  - PASSED - @admin archives a task and verifies Download Report and View Document in Archived table
  - PASSED - @admin creates, updates (categories + passcode), and deletes a unit
  - PASSED - @admin creates, edits, toggles role, and deletes a user
  - FLAKY - @admin resets a user password and can view dept temp password
- UNSTABLE ✅ Pass: 23 | ❌ Fail: 1 | 🟡 Flaky: 0 | ⏩ Skipped: 0 - Run Time: 3/8/26 6:23:35 AM ESTView RunView HTML Report
  Test Cases
  - PASSED - authenticate as admin
  - PASSED - authenticate as user
  - PASSED - authenticate as streaming user
  - PASSED - @admin creates, edits, and deletes a category
  - PASSED - @admin reorders created categories
  - PASSED - @admin uploads a new PDF with categories, verifies it, and deletes it
  - PASSED - @admin views existing document and renames it
  - PASSED - @admin archives a document, verifies filter behavior, and unarchives it
  - PASSED - @admin verifies SuperDoc PDF is hidden when “Show Hidden” filter is applied
  - PASSED - @admin sets default email addresses for uploaded doc, verifies persistence, and cleans up
  - PASSED - @admin error validation when fields are missing
  - PASSED - @admin error validation when email is invalid
  - PASSED - @admin error validation when password is incorrect
  - PASSED - @admin error validation when user email is not an admin
  - PASSED - @admin valid login and logout
  - PASSED - @admin toggle password visibility
  - PASSED - @admin can request a reset code and sees failure with dummy code
  - PASSED - @admin views document from a multi-doc sign off task
  - PASSED - @admin edits an existing sign off task (name & description)
  - PASSED - @admin downloads a sign off report
  - PASSED - @admin archives a task and verifies Download Report and View Document in Archived table
  - PASSED - @admin creates, updates (categories + passcode), and deletes a unit
  - FAILED - @admin creates, edits, toggles role, and deletes a user
  - PASSED - @admin resets a user password and can view dept temp password