cancel
Showing results for 
Search instead for 
Did you mean: 

Developer’s Blog: Early results of the smartform enhancements to SIMon

Alan_K
Deputy Mayor / Adjoint au Maire

As we approach just over three weeks of the launch of the smartform enhancement on SIMon as a way to submit tickets to our Moderator Team, let’s keep the conversation going on how things have gone! 

 

If you’re new around these parts, I highly suggest reading my previous blog posts (Part 1 & Part 2) to learn about the context and behind-the-scenes of the development of this exciting change to our virtual assistant.

 

Without further ado, let’s open up the curtain and check in on how things have gone!

 

How is SIMon performing?

 

Since launching the smartform enhancement at the beginning of the month, SIMon has had 15K conversations, with 70% of conversations being resolved within SIMon. Way to go SIMon! This was undoubtedly boosted by the “predictive suggestions” and “intros”  improvement that we silently launched at the same time of the new smartform features. Have you noticed this? As you start typing your question, SIMon will preemptively suggest what he thinks you’re asking, making the conversational experience easier to use than ever before.

 

simon.PNG

 

Effect on ticket handle times

 

SIMon’s tickets now account for just under half of all Moderator interactions. With the smartform enhancement, our goal is to do two things:

 

  1. Decrease the average time to resolution (the time between a customer contacting us and when they respond that things are working back-to-normal)
  2. Increase the % of tickets that are closed in 1 message

 

We’re glad to hear that a number of you have experienced faster response times when using SIMon, and we are working hard to make sure that this continually becomes the norm moving forward. SIMon’s tickets are indeed prioritized ahead of regular private messages to encourage more of our customers to try this new process (see: Why should I use SIMon?). To date, customers who use SIMon have a 20% faster time to resolution compared to private message users.

 

We are also observing a 10% lift in the share of tickets closed in 1 message thanks to the information collected in the forms. We’ve taken some small steps to improvement so far, and rest assured we are continually learning and collecting feedback to take further steps.

 

Addressing some Q&A

 

Q: When does the smartform link expire?

  • A: The smartform link is a unique URL generated by your chat session. It expires when any user submits a ticket using that URL. In other words, it is designed to be a one-use link for the original chatter, and not widely shared. That’s why if you click the same link once a ticket has been submitted through it already, you will see a 404 error as a result.

 

Q: What about a direct link to the form?

  • A: We are definitely aware of this ask and can appreciate it’s not ideal to tell a customer (who has posted on the Community already and shared their issue), to share it again with the Chatbot. That said, we also consider the need to deflect simple issues (which SIMon has proven to be very good for) and to not create an unnecessary workload to our Moderators. To that end, we will likely explore a middle ground solution to this but it is certainly one of the first items in our to do list (more on that in “What’s next”).  

 

Q: I can’t submit a ticket because of a 403 error message. What’s happening?

  • A: We addressed the following bugs at launch and in the recent weeks:

 

What’s next

 

We are constantly collecting feedback through an SMS survey that we send to customers who have contacted a Moderator since the launch of this initiative. Additionally, we are collecting feedback from our Moderators and Retail Partners in an ongoing huddle. We’ve started to accumulate a backlog of changes, including ones voiced by the Community about the preference for a direct link and live wait time.

 

I’m personally very excited for these early results as I know our Moderator Team has been learning and adapting to these changes as well. I’m hopeful we’ll continue to see additional benefits down the road when this new process becomes a well-oiled machine. What do you think of this change so far? Let us know down below.




Until next time,

Alan

19 REPLIES 19

pgermaine
Good Citizen / Bon Citoyen

I like how Simon uses key search words from the user to suggest solutions to questions. That in itself is helping solve questions faster and keep moderators more able to solve more difficult issues that Simon can’t. 

Alan_K
Deputy Mayor / Adjoint au Maire

@mimmo wrote:

@Alan_K can you make the bubble appear based on logged in status and or Commmunity level? 


Great idea! Not right now unfortunately, it's a universal turn on or turn off kind of feature.

flatl99
Great Neighbour / Super Voisin

Yay!! Got SIMon!!

mimmo
Retired Oracle / Oracle Retraité

@Alan_K can you make the bubble appear based on logged in status and or Commmunity level? 

Lets se simon 1.1 can do next time!  

Thank you! @Alan_K 

Alan_K
Deputy Mayor / Adjoint au Maire

@Korth wrote:

@cavemantoronto wrote:

SIMon. Can't stand him but can't live without him. Does Public Mobile know how much customers dislike it?  


I wouldn't say I dislike Simon. If the gadget actually helps people then it's worth keeping around.

 

What I do dislike are Simon's constant notifications. He occupies too much screen real estate on my mobile devices, it's especially frustrating when I can't even see what I'm typing because his big notification bubble is in the way. I've tried asking politely and demanding impolitely but PM insists on keeping Simon high priority and overlayed on top of everything, not even allowing mobile browsers/adblockers to disable Simon script without breaking the webpage. 

Please enable the user option to silence Simon until his services are required. Let Simon recognize requests for him to go away and sulk quietly in the corner, enable the option in community user profile, something, instead of investing efforts into doing the exact opposite.. Simon might become popular because Simon proves to be useful, Simon won't become popular because of manufactured celebrity status. 


Thanks for the feedback @Korth and @darlicious , I have also heard a number of Community members request the chat bubble to go away as it disrupted the mobile experience. I've removed that until we can figure out a better way to make SIMon more visible but not disruptive to customers!

Oh simple simple so very simple Simon. You are living up to your name sake. It is good to hear that he is resolving 70 % of inquiries. Obviously simple questions are getting resolved that is good. Takes the pressure.off both the moderators and the community. This leaves more time for the moderators so resolve issues that require expediency (no service) and issues only they can handle (accounts). This allows the community to help guide new users, troubleshoot, give advice or recommendations and just have lively discussions rather than constantly starting a conversation with "Reboot your phone" even though this is a common working solution. The constant pop up Simon is very annoying as is his location right by the post button. Changing these two irritants would go a long way in me despising Simon a little less....maybe.


@SD08 wrote:

Well, it's not surprising that customers who use SIMon have a 20% faster time to resolution compared to private message users, since they're being prioritized. When less than half of moderator interactions come from SIMon tickets, those who do use SIMon are effectively skipping past half the lineup. In that light, a 20% improvement seems...well...light.  It would be interesting to see where the average time to resolution ends up when ALL support requests are done through SIMon instead of just under half (i.e. when everyone is prioritized, then no one is prioritized, and there's no queue-jumping to skew the stats).


At least some of my Community online time is now spent seemingly telling people how to avoid/bypass SIMon to get a ticket opened. Even then, the simplest and fastest steps to accomplish that take a bunch of mouse clicks.

SD08
Retired Oracle / Oracle Retraité

@Alan_K wrote: 

.........

SIMon’s tickets now account for just under half of all Moderator interactions. With the smartform enhancement, our goal is to do two things:

...............

We’re glad to hear that a number of you have experienced faster response times when using SIMon, and we are working hard to make sure that this continually becomes the norm moving forward. SIMon’s tickets are indeed prioritized ahead of regular private messages to encourage more of our customers to try this new process (see: Why should I use SIMon?). To date, customers who use SIMon have a 20% faster time to resolution compared to private message users.

........

Alan


Well, it's not surprising that customers who use SIMon have a 20% faster time to resolution compared to private message users, since they're being prioritized. When less than half of moderator interactions come from SIMon tickets, those who do use SIMon are effectively skipping past half the lineup. In that light, a 20% improvement seems...well...light.  It would be interesting to see where the average time to resolution ends up when ALL support requests are done through SIMon instead of just under half (i.e. when everyone is prioritized, then no one is prioritized, and there's no queue-jumping to skew the stats).


@cavemantoronto wrote:

SIMon. Can't stand him but can't live without him. Does Public Mobile know how much customers dislike it?  


I wouldn't say I dislike Simon. If the gadget actually helps people then it's worth keeping around.

 

What I do dislike are Simon's constant notifications. He occupies too much screen real estate on my mobile devices, it's especially frustrating when I can't even see what I'm typing because his big notification bubble is in the way. I've tried asking politely and demanding impolitely but PM insists on keeping Simon high priority and overlayed on top of everything, not even allowing mobile browsers/adblockers to disable Simon script without breaking the webpage. 

Please enable the user option to silence Simon until his services are required. Let Simon recognize requests for him to go away and sulk quietly in the corner, enable the option in community user profile, something, instead of investing efforts into doing the exact opposite.. Simon might become popular because Simon proves to be useful, Simon won't become popular because of manufactured celebrity status. 

SIMon. Can't stand him but can't live without him. Does Public Mobile know how much customers dislike it?  

jor123
Town Hero / Héro de la Ville

Great news! Smiley Very Happy

Anonymous
Not applicable

@Luddite wrote:

As long as https://publicmobile.ca.ada.support/chat/ never fails I see no need for direct links to individual smartforms. Work on a system bug instead, please.

But OMG, 15k queries in a month is 500 topics a day!!! I take back everything I said about Simon be a waste of time. Robot Happy


I dunno. Web stats are easily inflated. If they had conversation path stats maybe. How many of those "hits" are all of us fiddling around with magic incantations trying to get a dang form 🙂

Or start to thumbs-up and done maybe. And not just the pm link to moderators or the mystical form. Those aren't resolution.

But then how many are people coming in and fiddling with it and throwing their hands up and taking their business elsewhere or suffering the problem.

We do see postings of people coming to the community saying they've had something going on for weeks.


@Anonymous wrote:

Gonna put the regulars here and the moderators there out of work. 🙂

 


Never fear, retail push will continue bring lots of folks here. Especially many more unfamiliar with online support as the only mode to get assistance. I may need a crash course in using Android OS to be of any help. Robot wink


>>> ALERT: I am not a CSA. Je ne suis pas un Agent du soutien à la clientèle.

sa7375
Town Hero / Héro de la Ville

"SIMon has had 15K conversations, with 70% of conversations being resolved within SIMon." is indeed remarkable. 
Image result for kudos

Luddite
Oracle
Oracle

As long as https://publicmobile.ca.ada.support/chat/ never fails I see no need for direct links to individual smartforms. Work on a system bug instead, please.

But OMG, 15k queries in a month is 500 topics a day!!! I take back everything I said about Simon be a waste of time. Robot Happy


>>> ALERT: I am not a CSA. Je ne suis pas un Agent du soutien à la clientèle.

Wow, way to go simon. Bravos to you.

Anonymous
Not applicable

Gonna put the regulars here and the moderators there out of work. 🙂

Wait times or place in queue. Not sure which I prefer. My gut says place in queue because one never knows how long a "call" will take.

Need Help? Let's chat.