Interviews serve a specific purpose – to help you cut through the avalanche of candidates at each step and get to a shortlist of those who present the most suitable fit to the role you are recruiting for.
I’ve worked with companies that did 2 rounds, 3 rounds, even up to 8 rounds of interviews to select candidates at different levels. There are numerous interview techniques being used by recruiters – depending on the role, the seniority and a host of other factors.
One thing, however, stands out. No matter the level of diligence you employ, interviews are a hit and miss. I’m yet to meet somebody that can confidently and irrefutably prove that every single one of their interview decisions have hit the mark.
Sometimes we recruit the wrong candidate, sometimes the right candidate at the wrong level, sometimes the right candidate at the wrong time – and so on. You only find out how good a decision you made weeks, sometimes months, later. By then, it might be too late. Either for the candidate or for you or both.
Let’s be fair to the candidate – they are dependent on the interviewer to make the right decision – a lot of the time. The smarter ones can see when joining a prospective employer can be a disaster. But then again, this doesn’t always happen. Candidates accept offers, mainly, if basic expectations are met. It is the interviewer’s responsibility to bring in the right candidates at the right level and at the right time.
“It is the interviewer’s responsibility to bring in the right candidates at the right level and at the right time.”
So, is there a pill that can cure your interview woes?
Of course not! If there was, there wouldn’t be a job market, would there?
Okay, that statement was a bit outlandish – but you get the idea. Interviews can only be so successful – where humans are involved, no decision can be 100% right.
Yet, you can certainly make things better. You can strive to improve the hit rate of your interviews – again with a lot of techniques, but also by asking the right questions of the candidate.
In this post, we’re going to talk about some right questions. Questions that you can ask candidates appearing for Quality Assurance roles. I will give you a list of Quality Assurance Interview Questions that you should bake into interviews for Testing roles in your company.
The right questions will bring out specific qualities, skills, experience that might otherwise not be apparent, and help you make more informed (and hence probably more accurate) decisions about candidate selection. These questions will also help you analyse the fit of a candidate to a particular role. With that, let’s begin…
#1 What Testing tools do you use/recommend, and Why?
This is one of my favourite Quality Assurance interview questions. And I’ve covered this question previously.
You and I both know that tools help remove repetitive, redundant and repeatable activities from your roster; that they improve the flow of information across the project, team and organisation.
Tools help you focus on the high value activities and automate the rest.
In that sense, yes, tooling is important.
Where people are concerned, everything can be tainted with emotions. Emotions are involved even with matters like selecting the right Test Automation tool for your team. Believe me, it happens.
#2 In an Automated world, how relevant is Manual Testing?
This Quality Assurance interview question helps you pick off the candidates that tend to exaggerate the importance or use of Automated Testing.
“There is no right answer to this question, and sometimes it can come down to your (the interviewer’s) own personal bias towards either technique. Now that is something to watch out for.”
The inexperienced or over eager candidate tries to provide a response they think will please you. Whereas the right candidate will provide a dispassionate analysis of the pros and cons of each technique, and how they can coexist without any conflicts.
There is no right answer to this question, and sometimes it can come down to your (the interviewer’s) own personal bias towards either technique. Now that is something to watch out for.
#3 What is the difference between Agile and Scrum?
A fundamental Scrum and Agile question. Yet, you’d be surprised at the answers you get with this one.
Many candidates struggle to articulate a clear response to this question. So, I sometimes use it to unsettle them. Such questions make the interviewees more alert and at the same you get to measure how well they react to being put in a box.
This question is usually a precursor to more prodding discussions about Agile and Scrum, and the other Agile development methodologies out there. A good interviewer doesn’t stop with asking about the differences. They will try to test how deeply the tester understands Agile methodologies.
#4 What are the first three things you’d do if you join us?
No candidate understands enough about your organisation (unless they are internal of course, but still) to give you an effective response. At the same time, they cannot NOT respond. Why would you consider anyone that cannot respond to a fundamental question like this?
That is why this is a good Quality Assurance interview question to ask. Because it tests the candidate’s ability to demonstrate their understanding of what your team, department, organisation does.
Good candidates conduct prior research about your company or team specifically, and use the interview to cull more detail about the specific challenges you are facing on a day to day basis.
While you can’t really expect world changing ideas in response, it is known to happen. Some of the best ideas I’ve picked up are from brilliant candidates that I interviewed for a role in my team.
So this question isn’t a blank bullet. It’s an opportunity for the candidate to create a great impression on you with their response. And if you catch an idea or two that can make your life easier, that’s a bonus!
#5 – How would you bring down the defects count in a project?
With Quality Assurance roles, you aren’t just looking for testers and test managers that can do run of the mill jobs. You’re looking for talented people to transform your ways of working and to infuse vigour and fervour into your team and your deliveries.
Questions like this one are designed to bring out the talent in your prospective Tester or Test Manager.
“Talent is about taking what you do and completely transforming it, so in the best case, you don’t actually have to do it anymore but still derive the expected outcome.”
Talent should attempt to bring sweeping changes to the way you do things. Sometimes such efforts can fail, but mostly however, they succeed. Personally, I believe ‘world changing’ ideas are worth a shot over ‘incremental improvements’. Don’t get me wrong, incremental improvements are the building blocks of Agile. Yet, now and then, the world changing transformations are necessary to take us into the future.
Think Apple’s iPhone with just one home button and multi touch in a world that couldn’t think of a smartphone without a keyboard and couldn’t get past awkward stylus-based touch screens.
Think Xiaomi in a world of Samsungs and Sonys that charged a bomb for quality and the rest that churned out subpar phones in the name of ‘affordable’.
That is talent. Talent is about taking what you do and completely transforming it, so in the best case, you don’t have to do it anymore but still derive the expected outcome.
When you ask this Quality Assurance interview question, you’re testing your potential candidates to see if they’re mere doers or innovators.
#6 – What is Agile, or What is Agile Testing?
Yes, this is a fundamental question. And yes, the answers are equally fundamental. Yet, there is more to asking this question than there is to any other.
Why do you need to ask this question?
Especially when you’re recruiting for the more senior positions in your team, asking this question can unseat the candidates a bit.
With experience, the average Agile Test Lead/Test Manager stops expecting such questions during interviews. They prepare for questions around the more complex Agile Testing topics – like exploratory tests, test-driven development, etc.
Probing your prospective candidates with this fundamental Quality Assurance interview question will push the best among them to pause and think about their response. More than getting an expected answer, this question is a good way to push them out of their comfort zone.
With this open-ended question, you allow the candidate an opportunity to express their Agile experience and thinking freely. In this way, this question also helps you measure the them on their ability to articulate their thoughts clearly.
#7: Does TDD guarantee a bug-free product?
With the advent of the more Agile concepts like Extreme Programming (XP), Paired Programming and Test-Driven Development (TDD), we found ways to build software better, faster and cheaper.
All these methodologies have simple goals – to speed up development, to discover major bugs earlier, and build a lot more product and faster than is possible with other methodologies. Extreme Programming, Paired Programming, Test-Driven Development and the likes do not give you a release-ready product. Release-worthy, yes, but Release-ready, No.
You need a level of (mostly Traditional) testing to make a product release-ready. By their very nature of focusing on the critical bugs, extreme Agile methodologies won’t (and can’t) give you this assurance.
#8: Explain how you decide on your Device Strategy?
For Digital-focused teams, device strategy is paramount. A sound device strategy will help you effectively manage product support.
- Does your app support Android Gingerbread? Really, you do?
- What about iOS 6?
- You should continue to provide support for Samsung Galaxy SII, right?
There’s much more to device strategy than some techie or tester arbitrarily deciding to regularly cull older devices, operating systems from support.
Device Strategy is a business decision.
You must consider questions like:
- What percentage of your customers still use a particular Android OS version?
- Are you going to try and support a small fraction of customers that choose to remain tethered to an archaic browser, mobile OS, device?
- Or are you going to nudge them to upgrade to a better phone, tablet, OS, browser?
Do your regulators have a say in your device strategy?
There are still a lot of firms that provide employees laptops with Windows 7 and Office 2010 or 2013. This is at times because their staff continue to use specific software that won’t yet work with Windows 10. Or, their IT security haven’t yet figured out how to fix all the vulnerabilities that a new OS version introduces to their internal systems and servers.
Device Strategy is a complicated beast, and is a decision not made lightly or by one individual. It’s a collective decision that has far reaching implications for your customers, colleagues and regulators.
Device Strategy plays a central role in determining the direction your team, project, product will take, and how this will impact customer experience and bottom-line. And a response to this Quality Assurance interview question should reflect that thinking.
#9 How do Testers work with Scrum Teams?
The answer is: It depends.
It depends on how agile a team really is.
The easiest answer is that the testers within the Scrum team, and deliver to sprints. This is true for well-run Agile initiatives. Embedding testers or testing roles within the Scrum team helps drive development to primarily pass test cases.
“The hallmark of a good Scrum tester is when they help reduce the defects count and percentage of defects found per line of code written.”
By nature of how Agile and Scrum work, a Scrum team is expected to possess all the skills necessary to deliver an Agile project — including Testing. And, we all know that anyone in the team can deliver any skill — when the team is truly Agile.
The hallmark of a good Scrum tester is when they help reduce the defects count and percentage of defects found per line of code written.
How do they do this? By working with the rest of the team to deliver to requirements. And what better way to do this than by working day-to-day as part of the Scrum team!
On the other hand, the more experienced agile tester knows that at times, traditional test cycles need to be run outside the Scrum.
When Traditional test cycles are (unfortunately) necessary
If you work for a large multinational company that is overhauling its mobile app for all its markets, they probably have a product team that is delivering some ‘core’ features for the app, to then be customised for individual markets before deployment. The ‘core’ team’s focus will be on fixing major bugs that block progress of product development; not make the product bug-free.
Depending on which version of the product is being deployed to each market, it is quite possible that a release needs to be thoroughly tested prior to deployment to ensure the code is almost bug-free. This is when you will need traditional System Integration Testing (SIT) and User Acceptance Testing (UAT) cycles to ‘clean’ the code to make it deployment-ready.
Unless somebody has worked primarily in self-contained projects where there is no dependency on teams external to the Scrum team, you should expect to hear about dedicated traditional testing cycles. Such knowledge demonstrates valuable experience in working on distributed Agile projects and working Agile alongside Waterfall teams.
This Quality Assurance interview question is important because it helps you measure the level at which a candidate operations, and the exposure they must working in non-ideal Agile working environments.
#10 What charts and graphs do you use to report Testing progress?
You expect a knowledgeable and considered response to this question.
The answer could, of course, be anything. Some candidates will have had exposure to the Burnup Chart, and its many uses and advantages over the Burndown. They could tell you about how they used Defect Trends to augment Testing Burndown reports, to present a more holistic view.
By challenging them to go beyond the routine answer, you can push boundaries with your candidate.
For instance, tell them you agree that the Burndown chart is a great reporting tool. So, you’re taking it off the table as an answer. Ask them what better reporting techniques exist.
For prospective Test Leads and Managers, this Quality Assurance interview question and its answer is a great way to measure experience and maturity. Ask for more detail on projects they led and reporting techniques they used (and why).
Now It’s Your Turn
Great recruitment decisions have been based on instinctive decisions made in the first five minutes. Then again, a fair percentage of such decisions have gone wrong. The most rigorous interview process in the world couldn’t save you from making a bad recruitment call.
We’re only human – and we recruit other humans. Strive to make your Quality Assurance interviews more productive by incorporating the questions above. And trust your judgement. If you don’t, who else will?
Do you have other Quality Assurance questions that you have found equally effective when recruiting? Share in the comments section below.