Third Quarter 2020 Benchmark Results for Mystery Shops and Resident Surveys Executive Summary

Since 1984, Ellis, Partners in Management Solutions has specialized in helping customers achieve their business goals. We are proud to offer a turn-key integrated customer experience program, backed by our outstanding customer service, cutting-edge technology, and longstanding ethical business practices. Our apartment mystery shops, resident surveys, live training, and e-learning partner (Edge2Learn) are resources that can be used independently to address specific needs or jointly for ongoing employee performance optimization.

In 1849, French writer Jean-Baptiste Alphonse Karr wrote, “The more things change, the more they stay the same.” While this pandemic has turned the multi-family industry upside down and inside out, one thing has remained the same – the customer is still the customer, and they have expectations. During the onset of the pandemic, they showed patience and forgiveness towards what would once have been considered under par sales presentations, but our data tells us that the tides are beginning to quickly turn. As companies continue to make efforts to train their teams to deliver the leasing presentation in new ways, including virtual tours and self-guided tours. The results show some pushback not only from the customers but from the employees themselves. Now more than ever, the feedback we receive from the customer must be carefully considered because they are our eyes and ears in this new pandemic world in which we all find ourselves operating. Are we making it difficult for our customers to do business with us? Are we missing the basics?

This quarter’s letter we begin to reveal answers through “Listening to the Voice of the Customer”.


Mystery Shops

Ellis benchmarks apartment mystery shop performance on the 10 key Benchmark questions and the Customer Experience. Companies can qualify for platinum, gold, silver, or bronze level based on their company’s overall Benchmark score for the quarter.

In 2019, the overall average Ellis Traditional Multifamily Industry Benchmark score across Ellis’ entire database of eligible shops was 85%. This rose slightly to 86% for First Quarter 2020. Our industry performance was moving in the right direction and then COVID-19 hit. As a direct result, during Second Quarter 2020, the overall benchmark fell 7 points to 79%. During Third Quarter 2020, the average rose back 83%, representing almost 4,000 in-person shops. These results reflect the efforts put in place to drive that number back up and are a testament to the resilience of our teams. The Ellis Customer Experience Benchmark score was 3.6 and remained steady at 3.6 for Third Quarter 2020.

In response to industry needs and customer demands, we introduced the Ellis Virtual Leasing Multifamily Industry Benchmark in Second Quarter 2020 based on 9 of the same key Benchmark questions plus an additional question on the effectiveness of the follow-up contact. In Second Quarter 2020, we conducted 3,243 virtual leasing shops and the average score was 56% (based on a 100-point scale). During Third Quarter 2020, the average Ellis Virtual Leasing Multifamily Industry Benchmark, representing 2,667 virtual leasing shops, improved slightly to 59%.

Due to COVID-19 circumstances, Ellis is not publishing mystery shop performance by Tier/Rank or Top-Performing Companies for Third Quarter 2020.

As we compare similar questions on the Ellis Traditional Multifamily Industry Benchmark and the Ellis Virtual Leasing Multifamily Industry Benchmark, we can see areas which are consistent in the delivery and areas that can be improved.

2020 q3 benchmark

Mystery Shops Overall Average Performance

benchmark summary

Resident Surveys

The Ellis Loyalty Benchmark identifies and recognizes the customer experience performance of companies subscribed to the Ellis Resident Surveys Program. It evaluates performance on 5 key touchpoints of the prospect and resident journey. The percentage of surveys responded to by the onsite team and the average number of days it took for the team to respond are also measured because these two factors impact the customer relationship and overall loyalty. Utilizing customer loyalty as a metric allows customer-centric companies the ability to forecast three specific customer behaviors: likelihood to convert/renew, willingness to pay more, and likelihood to recommend.

The overall average Ellis Loyalty Benchmark score in 2019 was 75%, and it increased slightly to 76% for Second Quarter 2020. In Third Quarter 2020 the score remains steady at 76%. In 2019, the onsite response rate averaged 88%, with improvement to 90% for Second Quarter 2020. Third Quarter 2020 dipped to 88%. The response time was 4.6 days in Second Quarter 2020 and slipped to 4.9 days in Third Quarter 2020.

Note: Ellis is not publishing Resident Surveys Top Performing Companies for Third Quarter 2020.

Resident Surveys Overall Average Performance

surveys performance

Ellis’ customer loyalty score is based on a scale of 0%-100% (see chart below):

loyalty score



Mystery Shops
How did we do?

The charts below reflect the average score of Ellis’ entire database for each of the 10 key benchmark questions and the Customer Experience by category, as well as the combined overall Benchmark scores for Third Quarter 2020. This data provides a roadmap to success, which in most cases means returning to the basics of sales and customer service.

mystery shops benchmark

Resident Surveys
How did we do?
The chart below reflects the average scores across all Ellis Resident Survey companies for each survey touch point for Third Quarter 2020, as well as the combined overall loyalty score and average accountability performance results.

surveys benchmark


While we witnessed a 4-point gain during Third Quarter 2020 on the Ellis Traditional Multifamily Industry Benchmark and a 3-point increase in the Ellis Virtual Leasing Multifamily Industry Benchmark overall for the quarter, our teams must continue to fine tune their skills in the leasing approach. To this end, it is time to turn our attention to Ellis’ Virtual Shopping Report and Ellis’ Self-Guided Shopping Report results. With marginal improvement in the benchmarking scores and self-guided tour comments showing too many unfavorable experiences, it is critical that we intentionally listen to the voice of the customer, learn from their feedback, and improve their journey.

1. The Virtual Tour

 What are they telling us?

During the COVID-19 pandemic, businesses of all types have been forced to quickly pivot to virtual selling. The best virtual tour can leave the customer feeling like they have just experienced the product in-person, yet an ineffective virtual tour can leave them feeling frustrated, confused, and without enough information.  Not all virtual tours are created equal when it comes to their effectiveness.

According to Ellis’ study of almost 6000 virtual shops conducted between March 2020 and September 2020, the different approaches to virtual leasing can impact the customer’s decision to lease or walk away.

effectiveness by tour

In the chart above, we see that the generic video is the tour of choice and possibly the new “easy button” for leasing professionals. We understand that in some leasing offices, the generic video is the only virtual tool available. Typically, passively pointing the customer to a webpage or generic video would not entice them, while actively walking them through a video and bringing the apartment to life through conversation can result in a positive experience. Interestingly, according to this chart, it is the personalized Face Time tour that is the most effective probably because it closely resembles the experience the customer would receive in-person. In the article, “The latest tool for selling real estate: FaceTime”, Kristen Olson, a home buyer, confirms our assumption, “I’d say to Julie, ‘Wait! Back up! Can I see that tile on the kitchen counter? Better angle!’ A lot of houses have virtual tours but with a virtual tour you’re kind of a passive viewer. With this, it really was like we were with her. The only thing we couldn’t do was smell the house.”

We know that virtual leasing will never replace the power of the face-to-face experience in the multifamily industry, but Pandora’s box is now open.  We expect that we will see more companies using and customers requesting the virtual option once the pandemic is over, so we must improve in this area.

What can we learn?

In the April 2020 article, “Realtors get creative to sell homes during COVID-19 crisis”, we get some additional insight into how our other real estate friends are handling the virtual tour, what it looks like, and how the customer is responding. FaceTime was mentioned in this article as a tool being used to make sure the customer gets to see every single detail of the house – the good and the bad – and not just a quick generic video.  Brittany Boyd, who referred to herself as a millennial customer in the post, agreed, “We buy a lot of things online. It really comes down to trust. I mean she’s right there zooming in on the cracks on the floor and walls.”                                                                          

If done well, the right virtual tour offers a safe and flexible option during a worldwide pandemic, and a talented leasing professional can use this tool to provide an extremely comprehensive tour of the community while building value and trust with the customer. Are we there yet?  Not quite.  We continue to see a significant disparity between the in-person experience and the virtual experience when comparing similar questions.

virtual vs in person

What is contributing to the low scores? The answers and lessons to be learned are often captured in the comments, stories, and sometimes granular details within the shopping report. In some cases, leasing professionals are making it difficult for the customer to do business with them.  Here is one shopper’s virtual tour experience:

“I was on a virtual FaceTime tour with a leasing professional. I was told that I would be shown a different floorplan than the one I was interested in. When the virtual tour began, the leasing professional walked around and showed me the outside of the buildings and the clubhouse which was dirty from a party the night before, yet he made no comments about the condition. He walked around and said nothing.  I had to initiate the conversation. He never explained to me why he was not able to show the apartment. When the tour was over, he asked me if I had any questions. THE END.”

It is unlikely this customer would have leased, but is this just a unique situation? To get a better understanding of the overall virtual experience based on Ellis’ mystery shops, we have summarized the common themes based on Third Quarter 2020 results below:

There appears to be a hesitation to walk customers “virtually” through the website in-spite of being prompted. Virtual leasing seems to be awkward and unnatural to many leasing professionals. In fact, the onus appears to be on the customer, who in many cases is having to ask for a “true” virtual tour. Unfortunately, a popular virtual leasing approach is, “We have a video that does the virtual tour on our website”, or “After you look at the video/pics, give me a call.” It is more of a self-service approach than an intentional way to engage the customer in a tour.

There is the appearance of leasing professional apathy as well as frustration and exhaustion. This could be contributing to leasing professionals not answering the telephones, keeping their appointments, sending information to the wrong email address or not at all, appearing distracted, and in some cases, just not being nice to the customer.

How can we improve the customer’s journey?

Customer feedback is not always pretty but we can learn from it, and we can improve their journey by taking action. It is important for companies to self-evaluate and measure the right things with their external and their internal customer. We have provided and will continue to provide very transparent insight from the external customer, but it is up to leaders to take actions that will improve their customer’s journey. This could simply mean slowing down, giving attention to customer feedback, and asking the right questions.

Start with this question: “What kind of virtual tour will encourage customers to trust us and choose our community over our competitor? What does it look like?”


  • Do we have the tools and technology in place to deliver such an experience?
  • Are our teams trained and prepared to use the technology? Check out our FREE virtual training.
  • Do we have a standard to follow which will vary depending on the customer’s needs? FaceTime virtual tour.  B. Walking them virtually through the website.  C. Screenshare (i.e., Zoom, GoToMeeting).  D. Personalizing a video based on the customer’s needs.  E.  Reviewing a generic video with the customer.

Then Dig Deeper…

  • Do we have the right people in the right positions?
  • Is the new customer simply a distraction in the day? Has this pandemic turned leasing into just one more thing to do?
  • Are our employees overwhelmed with customer complaints and demands now that many residents are occupying their apartment 24/7? How can we provide some relief?
  • Do we have enough office coverage?
  • What accountability is in place for poor performance? Is the poor performance the result of a can’t or a won’t?
  • What distractions must be removed so the leasing professional can focus on the needs of the customer? Are the distractions personal or work related?

An effective leasing professional is capable of transferring their knowledge and skills from the traditional leasing presentation to the virtual presentation – Establishing Rapport, Identifying Needs, Presenting Solutions, Closing the Sale, Following Up.  All of these techniques are still in play. While the same elements are required, new skills must be acquired to make best use of websites, generic videos, FaceTime, Zoom, etc. so they appear seamless to the customer. There is not an overnight fix. It takes time. It takes patience. It takes training.  It takes practice.  It takes attention.  At the end of the day, what is important to you as a leader will become important to your employees.


2. The Self-Guided Tour

What are they telling us?

Self-guided tours make it faster, easier, and more convenient to view apartments. While self-guided tours allow more flexibility, they also minimize opportunities for teams to positively impact the sale and deliver a great customer experience. Historically, customers who experience a personal interaction convert at higher rates than those who do not, showing that a knowledgeable leasing staff clearly makes a difference.

We saw some pushback to the self-guided tour with roughly 80% of shoppers this quarter responding unfavorably to their self-guided tour because there was no one available to answer questions – to help with directions to the apartment, furniture placement, general leasing questions, etc. On the flip side, a recent article, “How Is The Multifamily Market Dealing With COVID-19 And What Tech Is It Turning To For Solutions”, paints a very positive picture of self-leasing and the goal to make it as easy to rent an apartment as it is to book your next vacation. What will the future look like? Will customers prefer to forego a visit to the community and simply register to view remotely, self-tour, and transact their lease online? Will this be a generational divide? Only time will tell but there are certainly pros and cons when it comes to this alternative and how the customer will respond.

The self-guided tour as it currently stands is another “easy button” for leasing professionals. Some customers prefer the autonomy and privacy while others find self-guided tours frustrating and lacking. The self-guided tour has certainly been a viable option for leasing teams during this pandemic, but how will it perform once we come out on the other side? Can we get this right so that it offers both a flexible and effective option for customers?

What can we learn?

Since the self-guided tour is mostly a self-service option, the majority of the data comes to us in the form of written customer comments. Here is one shopper’s Q3 self-guided tour experience. In some ways, it closely resembles a Sherlock Holmes mystery.

“I was told on the phone that I would experience a traditional tour – a guided tour. When I arrived on property, I was greeted through a window and was informed that I would be taking a self-guided tour. I was confused on what was happening. Through the window, I could see several employees standing around inside the office. The leasing professional slipped information through the mail slot. The leasing professional and I had a brief conversation by phone with her on her cell phone as seen through the window and me standing outside the office. I was given keys and sent on my way. I attempted to find the apartment based on the information I was given but I could not locate it. I repeatedly called the office seeking help, but no one answered. I went back to the office and dropped off the key and left. THE END.”

To get a better understanding of the overall self-guided tour experience, we collected shopper feedback based on Ellis’ Q3 mystery shops and summarized below:

The self-guided tour offers an easy option for the customer and the leasing professional. The customer can tour on their own time and on their own schedule. It is a time saver. It is low pressure. This is a popular option for many leasing professionals who are quick to encourage customers to “go look” and call them if they are interested. Rarely, is there an attempt to close the sale or follow-up after the customer’s visit. Some leasing professionals join the customer in the apartment using FaceTime or a call while others provide Alexa in the apartments to answer their questions. Because of the absence of a leasing professional, the experience can come across as very impersonal. There is truly no experience. Occasionally, a leasing professional will make themselves available in the office to answer questions but time and time again, offices are closed, and phones are not being answered. A few leasing professionals choose to accompany the customer to the apartment and wait outside the door, which has resulted in the leasing professional using that time to talk and text on their cell phone with friends, family, etc. The entire self-guided tour is awkward right now. There is very little to no human contact and the burden to communicate is left to the customer.

How can we improve the customer’s journey?

As we mentioned before, it is not always a pretty picture but customer feedback gives us a goal upon which to focus our efforts. We can improve the journey by evaluating current self-guided touring processes and taking action. It is obvious that changes need to take place. While some companies will launch elaborate tools to make this a common and effective touring option for their customers, others will use it to simply fill in the gaps. At the very least, we can do a few things right now to improve this experience for the customer.

Here are a few ideas to consider.

  • Clearly define the self-guided tour process and expectations for the onsite teams and customers.
  • Establish if and how the leasing professional is going to engage with the customer before, during, and after a self-guided tour occurs.
  • Ensure the onsite teams are trained on self-guided tours.
  • Provide clear and detailed materials to the customer. The customer does not know your community the way you do.
  • Remember, a self-guided tour for the customer is also their experience.


  • Be professional, knowledgeable, and helpful always.
  • Be available while the customer is in the apartment. The customer should have a direct line to the leasing professional while they are onsite and especially during business hours.
  • Invite the customer back to the office or use Face Time or a call to answer any questions.
  • Attempt to close the lease.
  • Follow up in a manner that ensures the leasing conversation continues to move forward.


  • Do not make the self-guided tour your first choice until the process has been perfected.

We can learn a lot from our customers if we listen. As the market tightens and becomes more competitive, it is critical that we intentionally seek out their feedback and listen to their voice. Everything feels difficult right now but there will be a time when we move past this pandemic and reflect on all of the lessons we have learned and the new ways of doing business that have been adopted into our world. While some may be temporary others might stick, it is critical that our leasing professionals get to know them and perfect them. There will never be a replacement for the face-to-face engagement, but it is likely that virtual tours and even the self-guided tour will continue with us in a big way long after the pandemic has ended.

Listen to the voice of the customer.

Thank you for your ongoing support, participation and feedback, which help make this report informative, fresh, and a reliable resource. We hope you will find Ellis Partners in Management Solutions to be not only the finest source for apartment mystery shopping and resident surveys but also a training resource (including our partner Edge2Learn) for your organization. Additional support and information can be found on our website. Also, our free multifamily training resource library includes several virtual leasing training resources for you and your teams.

October 30, 2020

Prepared by Joanna Ellis, Chief Executive Officer and Francis Chow, Chief Strategic Officer