​

What is response testing for?

We use response testing to bulk test different outcomes that might occur, depending on which customer you wish to simulate.

For example, you may want to test one workflow that has many different Outcomes. In an order status workflow with different user contexts and order statuses, there’s many different combinations you would like to quickly review to see how the workflow would perform when live:

• orderStatus == “fulfilled”, customerTone == “angry”, or
• orderStatus == “delivered”, trackingLinkUrl == exists

Each output of a tool (e.g. ‘fulfilled’ or ‘delivered’) is unique depending on the order history of the customer contacting your support team.

Response testing allows you to specify which path of the workflow you want to test so you can quickly understand if your configuration will work across different customer situations.

​

How does it work?

Go into Response and select a specific workflow.

You can either select to test the live or draft version. We often test on draft to identify if the new workflow version will perform better than the live one.

​

The sample

Download the sample test cases so you have a view of what inputs you need to run response testing. These are the different columns you need to input:

id: You can name the id anything that will help you identify what this line item is testing (e.g. “Scenario A” or “Is customer, No orders”)

firstMessage: Input a first message from a customer you want to test. You can choose to add this from a live ticket as a back test, or create your own first message samples.

tools: For the path that this workflow will follow, specify what the outputs will be for each tool. You only have to do this for variables that have a purple outline around it. The easiest way to get this information is to look at past output data from existing tickets by clicking “View logs”.

If you’re not familiar with JSON, you can use JSON online validator.

1. Copy paste this in as a starting point.

   [
       {
           "toolName": "",
           "outputJson":
       }
   ]
   

2. Go into the tool you’re trying to simulate the output of, and copy the output

3. TKTK

4. If you want to add more than 1 tool, you can comma separate it:

   [
     {
       toolName: 'getCustomerByEmail',
       outputJson:
         '{\n  "id": "clf97y85u01k7nn01aesn176b",\n  "email": "tonigunncopy@gmail.com",\n  "lastName": "Gunn",\n  "firstName": "Toni",\n  "customerId": "clf97y85u01k7nn01aesn176b",\n  "toolOutput": {\n    "0d368422-9acd-42e6-857c-3fe9a636f2e6.xnocEnXIB6RHcTLPVr8nbd+Oqno06V1alads9YwiIEA=": {\n      "id": "clf97y85u01k7nn01aesn176b",\n      "email": "tonigunncopy@gmail.com",\n      "lastName": "Gunn",\n      "firstName": "Toni",\n      "customerId": "clf97y85u01k7nn01aesn176b",\n      "addressString": "8 Kolora Way Ocean Shores NSW 2483 Australia",\n      "preferredName": "Toni"\n    }\n  },\n  "addressString": "8 Kolora Way Ocean Shores NSW 2483 Australia",\n  "preferredName": "Toni",\n  "Yes (email exists in admin)": {\n    "test": "Scenario Yes (email exists in admin) passed: Assert {customerEmail} is truthy. AND Assert {email} is truthy.",\n    "result": true\n  },\n  "0d368422-9acd-42e6-857c-3fe9a636f2e6": {\n    "id": "clf97y85u01k7nn01aesn176b",\n    "email": "tonigunncopy@gmail.com",\n    "lastName": "Gunn",\n    "firstName": "Toni",\n    "customerId": "clf97y85u01k7nn01aesn176b",\n    "addressString": "8 Kolora Way Ocean Shores NSW 2483 Australia",\n    "preferredName": "Toni"\n  }\n}',
     },
     {
       toolName: 'getCustomerByEmail',
       outputJson:
         '{\n  "id": "clf97y85u01k7nn01aesn176b",\n  "email": "tonigunncopy@gmail.com",\n  "lastName": "Gunn",\n  "firstName": "Toni",\n  "customerId": "clf97y85u01k7nn01aesn176b",\n  "toolOutput": {\n    "0d368422-9acd-42e6-857c-3fe9a636f2e6.xnocEnXIB6RHcTLPVr8nbd+Oqno06V1alads9YwiIEA=": {\n      "id": "clf97y85u01k7nn01aesn176b",\n      "email": "tonigunncopy@gmail.com",\n      "lastName": "Gunn",\n      "firstName": "Toni",\n      "customerId": "clf97y85u01k7nn01aesn176b",\n      "addressString": "8 Kolora Way Ocean Shores NSW 2483 Australia",\n      "preferredName": "Toni"\n    }\n  },\n  "addressString": "8 Kolora Way Ocean Shores NSW 2483 Australia",\n  "preferredName": "Toni",\n  "Yes (email exists in admin)": {\n    "test": "Scenario Yes (email exists in admin) passed: Assert {customerEmail} is truthy. AND Assert {email} is truthy.",\n    "result": true\n  },\n  "0d368422-9acd-42e6-857c-3fe9a636f2e6": {\n    "id": "clf97y85u01k7nn01aesn176b",\n    "email": "tonigunncopy@gmail.com",\n    "lastName": "Gunn",\n    "firstName": "Toni",\n    "customerId": "clf97y85u01k7nn01aesn176b",\n    "addressString": "8 Kolora Way Ocean Shores NSW 2483 Australia",\n    "preferredName": "Toni"\n  }\n}',
     },
   ];
   

This is the correct format: [toolName:string]:string, [outputJson:string]

Here are examples that would work:

• messageAfterPaused: This only requires inputs if there are Gather nodes in your workflow. The message after paused simulates what how the customer would respond after a Gather node.

• This is the correct format: { [nodeDisplayName:string]\: string }

• Here are other examples that would work:

  • {"confirmCurrentShippingAddress":"No I moved to 420 colony cove dr, san jose, ca 95123"}
  • {"Is your account closed?":"Yes it is"}

• tags: Specify the segment, this would usually be default.

• expectedResponse: This is for your reference to see what the expected response should have been, to give you a comparison to what the AI actually outputs

• expectedActions: This is for your reference to see what the expected response should have been, to give you a comparison to what the AI actually outputs (e.g. ApplyTag, ReassignToGroup)