Before You Ask AI to Shortlist Candidates, Define the Job

A hiring manager sends you a folder of CVs and asks for a shortlist by lunchtime. An AI tool offers to summarise every application and rank the candidates. For someone starting in HR, that sounds like a welcome shortcut.

But what exactly is the tool judging? Whether candidates mention certain words? Whether they have done similar work? Or whether their CVs simply sound impressive?

A research preprint submitted on 12 October 2026 makes this a timely question. Called EHR-Bench: A Chinese Multi-Task Benchmark for Recruitment AI, it evaluates language models across different recruitment tasks. The researchers report that models with similar overall results can have different strengths on individual tasks. Performance also varies with the source material and candidate pool. This is research on Chinese recruitment tasks, rather than proof of performance in Indian hiring. Its practical implication is useful: test the particular job you want an AI tool to do before trusting its output. [1]

For an HR student, that means learning to separate three activities: extracting information, interpreting evidence and making a selection decision. A system that neatly lists qualifications has not necessarily established who can perform the role.

Start by getting the job clear

Imagine a Kochi-based business recruiting a junior recruitment coordinator. This is an invented learning example, not a Bayspro employer or placement story. The employee will arrange interviews, maintain a candidate tracker and send accurate updates to applicants and managers.

Before reviewing applications, ask the hiring manager what successful work would look like. Perhaps the coordinator must spot scheduling clashes, maintain complete records and write clear messages. Those requirements give you something concrete to assess.

“Confident personality” is much harder to interpret. So is “good background”. Ask what observable work those descriptions are supposed to predict. If a requirement cannot be explained in relation to the job, it needs another discussion.

Look for evidence in each application

Now consider two fictional applicants.

Anjali’s CV describes coordinating a college placement drive. She maintained a spreadsheet, followed up with participants and rearranged interview slots. Rohan’s CV mentions an internship at a recognised company and describes him as an “excellent communicator”, but gives little detail about his responsibilities.

Neither application settles the hiring decision. Anjali’s experience gives you relevant questions to explore. Rohan’s missing detail gives you questions too. The useful next step is to investigate what each person actually did.

An AI summary might help organise that information. Give it a narrow task: identify evidence against the agreed requirements, point to the relevant CV passage and mark information that is missing. Ask it to distinguish the applicant’s own claim from a demonstrated example. Avoid treating a numerical “fit score” as an explanation.

A simple review sheet can have four columns: job requirement, evidence supplied, question to ask and reviewer’s note. For Anjali, the scheduling row might say: “Placement-drive coordination mentioned; ask how she handled two interviews booked at the same time.” For Rohan, it might say: “Scheduling experience not described; ask whether his internship involved appointment coordination.”

That wording matters. “Not described” leaves room for clarification. “Cannot schedule interviews” makes a conclusion the application does not support.

Give candidates a fair practical task

The next stage should give both applicants a fair opportunity to demonstrate the work. CIPD’s selection guidance, updated on 22 September 2026, recommends structured interviews with consistent questions and scoring criteria. It also advises using assessment tasks that resemble the actual job. This is professional practice guidance, not a statement of Indian employment law. [2]

For our fictional coordinator role, prepare a small interview timetable containing a clash and one incomplete entry. Ask candidates to identify the problems and draft a rescheduling message. Use the same instructions and agree beforehand what a good response should contain: the right date and time, a clear explanation, a polite tone and a request for confirmation.

If the job includes using AI, you could add a separate exercise in which candidates check a deliberately flawed AI-generated email. Explain what assistance is permitted. You are assessing their ability to notice and correct mistakes, so the exercise should make that expectation clear.

For classroom practice, use fictional CVs and invented contact details. In a workplace, follow the organisation’s approved process for handling applicant information and using AI tools. An interesting experiment is not a reason to upload someone’s application into an unapproved service.

Practical takeaway

Build one evidence sheet before generating a shortlist. For every recommendation, record the job requirement, supporting evidence and question still unanswered.

Keep that sheet, the sample task and your assessment notes in a student portfolio. They can show how you think about recruitment beyond arranging interviews. To develop these skills further, explore Bayspro’s Diploma in HR Management, Payroll & People Operations, or the PG Diploma in Human Resources Management.

Sources & further reading

1. EHR-Bench: A Chinese Multi-Task Benchmark for Recruitment AI

   Research preprint submitted 12 October 2026. Chinese recruitment benchmark; not validation of Indian hiring decisions.

2. CIPD — Selection methods

   Updated 22 September 2026. Professional guidance on structured interviews and work-related assessments.

Put your interest into practice

Explore a relevant programme.

Diploma in HR Management, Payroll & People Operations ↗PG Diploma in Human Resources Management ↗

Have questions? Talk to a programme adviser.