Key highlights
- Discover how to set up effective UAT with realistic environments and clear goals.
- Understand the role of user feedback in building better systems.
- Get actionable tips to make UAT a valuable step in your release process.
At 4 am, our team assembled to complete a deployment to our new platform. We migrated the operations services from the legacy monolith to our in-house microservices application. Everyone was excited about this migration. Our customers were already using our new infra and we could see an improvement in their experience. The benefits we could reap from migrating our Operations to the same system had no roof.
We deployed all the services as per the deployment plan. We monitored the system for an hour. The orders were flowing in, and all systems were a go. We handed over to the hyper-care team and went to take a well-deserved rest.
In less than an hour, we started noticing several messages on my Slack. The overwhelming message I was getting was that the system was not working. We checked all the services and the system parameters were all good. All the services from staging were deployed as is on prod.
I joined one of the hypercare calls to understand more. It became clear to me that the product team had a different understanding of the workings of the operations than the operations team. While the system looked like our previous operations portal, the system behaved differently. A lot of hacks had been built into our monolith. The hacks allowed the operations team to cruise through their day. These were not captured in our product requirements.
This is where I believe a UAT (User Acceptance Testing) phase would have been beneficial. I had proposed the stage of UAT, but it wasn't a practice at the organization. Since we had to move to our new infra, everyone agreed that what they saw in staging was good enough to go live.
Introduction
Software testing has matured in leaps and bounds in recent times. With the introduction of automation at various layers, a lot of testing is now run time and is very efficient. There is still scope for testing the application manually. The majority of manual testing is visual testing. Functional testing can be validated by automation scripts.
There is another aspect of software testing, which has taken a backseat in recent times. Especially since the introduction of microservices. O'Reilly's "Microservices Adoption in 2020" report found that 77% of respondents had adopted microservices, with 92% experiencing success. Microservices have the ability to deploy instantly to production. Testing has largely been seen as a blocker on the deployments rather than a value addition in such environments.
As a release manager, I have been in this boat many times. The testing team hasn’t signed off yet and the developers are trigger ready to deploy the solution to production. Here, context is important. There are scenarios where we could learn more by releasing it to production than by testing it on our environments. But a good number of cases always benefit from conducting a well-planned and thought-of UAT before deploying to production.
UAT - testing or test drive?
I would argue that UAT is not really a testing of the system. In a lot of cases, the system has been validated for performance and core functionality by the testers and developers.
When you go to buy a car at the showroom, you take the car for a test drive. We aren’t testing the car for its engine brakes or tires. We are getting a feel of how it is to drive the car. That is what UAT is in my opinion. The tech teams would resolve any bugs like the car manufacturer would.
How to UAT?
The foundation: who, where, and what
Who
The first step is always to understand who are the participants of this UAT. For internal users, this is straightforward.
In the case of a customer-facing application, a user research team and your customer-facing teams are your best friends.
In both cases, you should identify the impact of the changes you are deploying.
- In the case of internal stakeholders, identify the roles impacted by the change.
- In the case of a customer-facing app:
- Segment the customer base by those impacted.
- Identify the user persona impacted.
Where
Once this is sorted, work with your infrastructure teams to prepare a suitable environment for UAT. The environment is where the UAT would be conducted. This could mean reusing an existing environment or building one for these purposes. The decision can be taken based on the scope of the change. A few pointers to keep in mind is:
- Data: The environment should be as close to how the production environment would feel. For eg: Your operations team could have customers and locations like how they would find in production.
- Integrations: Try replicating as many integrations as possible. The customer should feel like they are using the final product. For eg: In the case of mobile apps - prepare a prod app pointing to the updated services
The Golive App is designed to assist you in identifying all the essential components for your User Acceptance Testing (UAT) through its innovative Dependency Map feature.
Additionally, understanding the version available in each environment and tracking the deployment of your Jira tickets using the Golive Issue Panel is important for effective release management.
If you’re a release manager looking to improve your processes, our Release Management: Essential Guide provides valuable insights into managing and coordinating releases effectively.
What
One of the reasons for the failure of UAT can be the lack of focus. The customers do not have time to test everything. It is then essential to identify the scope and create a plan for the same.
- Identify the test cases you want the customer to focus on. Also important is who is preparing the test cases. Your customer support team or application champion might be better suited than your QA teams.
- Identify the time of testing. This is important because usually setting up the UAT environment is expensive. Hence monitor the testing schedule and manage the environments.
A test management tool, such as Xray Test Management for Jira, can assist your team in clearly defining the scope of UAT. Additionally, the Golive App for Jira will help you locate the most suitable test environment for your UAT, which will be available within your required timeframe.
Stick to the plan, but don’t get stuck with it
From the many UATs I have conducted, I have seen that the development teams get stuck to the scope too much. Some of the issues that the customer is raising could be trivial in the mind of the application teams. These could be a major source of discomfort for the customer.
For example, one of our operations portal features was a simple flag on delayed orders. The application team was working on a full-fledged service for that feature. They gave a table with delayed orders on another page. In theory, it solved all the problems, but it took the intelligence of the system away.
Seeing a delayed order on the map, gave information about the order at a glance, which was lost to the operator on production. Though we are talking about a simple click, it could mean a lot of pain during the day, especially if we are expecting peak volume.
This information and feedback are vital parts of the UAT process. While the customer will sign off on the application, the intelligence shouldn't be lost.
Treat it like production
This is my final piece of advice on conducting UAT. Set it up like it is production. Have your support teams take UAT defects instead of a product manager. Some of the issues or workarounds can be resolved by the support teams instead of the development team. This will build confidence for the customers. Should they encounter issues on prod, the framework is robust enough to manage the changes. Also, if it doesn't, this is valuable early feedback for the organization.
In recent times, UAT has been ignored because of its association with waterfall methodologies and phased models of delivery. Call it user feedback or anything else, it is absolutely worth slowing down for.
Conclusion
User Acceptance Testing (UAT) is a vital component of any successful deployment. It ensures that the system not only meets technical requirements but also aligns with real-world user workflows and expectations. Skipping this phase, as shown in the example, can lead to disruptions that could have been avoided with early feedback. By engaging the right participants, preparing realistic environments, and focusing on user-centric outcomes, UAT becomes more than a testing phase—it becomes an essential step in delivering value.
Key Takeaways
- Identify key stakeholders and participants early to ensure comprehensive feedback during UAT.
- Set up environments that mirror production as closely as possible to reflect real-world conditions.
- Clearly define the scope and priorities for UAT to focus user efforts on high-impact areas.
- Treat UAT like production by involving support teams to replicate real-world issue management.
- Use feedback gathered during UAT to refine processes and align the system with user expectations.
By implementing these strategies, teams can turn UAT into a powerful tool for building confidence and delivering effective, user-friendly solutions.
Download the Release Checklist
Get access to the Google Sheet used by our Release Management community.
Apwide will use the contact information you provide to share updates about our products and services, which may include relevant offers from trusted partners. You can unsubscribe at any time. To learn more about unsubscribing and how we protect your privacy, read our Privacy Policy.
