The Future of AI in Competency Assessments

AI: the buzz word of the decade, the closest we’ve come to real world magic, the solution to all our problems.

Since Open AI first debuted ChatGPT in November 2022 everybody and their mother has scurried to invent new and fantastical ways to employ the use of AI. Now, it’s everywhere: in education, healthcare, and, as I recently discovered, even in fridges. (An AI-powered beverage centre, of indeterminate use)

And while we might all be having a great time with it, firing out emails in record time and consulting AI as an on-demand therapist, a divide is starting to emerge between AI that genuinely adds value, and AI that simply looks intelligent.

To understand that divide, it helps to demystify what sits behind AI systems powered by large language models (LLMs). And since this article is published by a global competency assessment company, its employee feedback technology, not smart fridges, that we’ll focus on here.

Halo Feedback has already launched two iterations of an AI-powered Virtual Coach. Our latest version, runs seamlessly throughout the reporting process and transforms how users interact with their assessment results, offering real-time insights, contextual intelligence, and simplified data analysis It has lots of nifty little features designed to improve comprehension and make it easier for both the job holder and their manager to highlight key points of interest, to then drive behavioural change and growth.

Like ChatGPT, it runs on an LLM, but within clearly defined boundaries, ensuring all outputs remain grounded in the assessment and framed through professional reporting.

In our humble (and completely unbiased) opinion, our AI Virtual Coach works brilliantly in the reporting stage of competency assessments. But as AI continues to develop and push boundaries, so too should we. A natural next step for us then, is to explore how AI might be used earlier in the process, specifically in how 360 feedback is collected and interpreted.

There are two main paths to go down:

• A prompt-driven approach, where an AI chatbot controls the flow of questions
• An engine-driven adaptive approach, where AI enhances the experience but remains governed by the assessment platform

At first glance, this distinction can feel minor, it certainly did to me the first time I heard about it but in practice, it determines whether the data you collect is explainable, comparable,and ultimately trustworthy.

A prompt-driven chatbot-led survey would likely feel natural and adaptive. It’s also relatively easy to build (we’ve explored it ourselves). The problem is that competency assessments aren’t conversations for their own sake, they’re structured tools designed to measure behaviour.

When AI controls the flow of questions, standardisation quickly breaks down. Conversations drift, depth varies between raters, and behaviour coverage becomes inconsistent. The result is data that looks rich, but is difficult to compare, validate, or turn into coaching-grade insight.

The alternative, and more complex route is an engine-driven adaptive 360, where AI listens and interprets, but doesn’t decide what matters.

This is the idea behind our new feature, Adaptive 360.

Rather than replacing the assessment logic, Adaptive 360 would use AI to support it. Halo’s engine, guided by structured competency data, remains in control. The role of AI is to analyse how people respond; identifying clarity, uncertainty, strengths, and potential risks, and feed that understanding back into a rules-driven process.

Think of it less like an open-ended chatbot, and more like a skilled interviewer who knows exactly what they’re listening for.

When a participant responds, AI helps interpret the language being used. From there, Halo’s engine determines what happens next: whether a follow-up question is required, whether a behaviour has been sufficiently evidenced, or whether it’s time to move on. Every question has a clear rationale, and every step can be traced back to defined competency data.

Crucially it is behaviours, not conversations, that remain the unit of assessment. Any adaptive questioning has an explainable origin. For example, a follow-up might be triggered because multiple peer raters highlighted a potential risk in decision speed, not because the AI decided to “dig deeper”. Similarly, if several raters provide vague feedback about a leader’s vision, Adaptive 360 may prompt for a concrete example, because that behaviour hasn’t yet been clearly evidenced.

The result is a survey experience that feels human and responsive, without sacrificing consistency or fairness. Participants aren’t burdened with unnecessary questions, and organisations still receive data they can trust, compare, and act on.

In other words, the intelligence isn’t just in the AI, it’s in how it’s being leveraged.

At Halo, we believe AI should enhance the user experience without undermining assessment integrity. That sometimes means taking the more complex path, but it’s the only way to ensure AI delivers insight, not just spectacle. While Adaptive 360 doesn’t yet have a launch date, it represents our commitment to using AI in a way that’s adaptive, explainable, and grounded in what competency assessment is designed to do.

 

Published by Halo Feedback. Written by Erin Power