AI Bias

What AI Bias means

AI bias is systematically unfair results from an AI system. The word that matters is “systematically”. A single wrong answer is an error. Bias is an error with a direction: the same kind of person, subject or viewpoint is treated worse each time, and the pattern is invisible from any one result.

The site’s example is hiring, where the harm is easiest to see. A tool that screens out qualified candidates does so quietly. Nobody sees the people who were not shortlisted, so nobody sees the pattern. In hiring it is a named concern, and recruitment and candidate-evaluation AI is classified as high-risk under the EU AI Act.

The site’s short version is that a bias subject has to be named in a curriculum, not assumed to be covered by teaching the software. Tool training alone does not close the gap. This page covers where bias comes from, what the HR article reports about it, what the law says, and how to keep a person in the decision.

Where Bias comes from

The generative AI page gives the plain account. A model reflects whatever bias existed in the material it was trained on. If that material skewed in a particular direction on a particular subject, or represented some groups, languages or viewpoints more than others, the model’s output can skew the same way. The page adds that this is not something a user can see by reading a fluent, well-punctuated answer. It takes a deliberate check against the question actually being asked.

The CV page makes the same point from the hiring side. Beyond the criteria a business states, a model invents its own from its training data, and that is where bias shows. The page gives the examples of filtering out women or people from poorer areas. Two lessons follow. First, a tool can apply criteria nobody wrote down. Second, removing names from a CV is not enough, a point the site’s HR training slide makes in its list of risks under “no names is not enough”.

None of this requires bad intent by anyone. It follows from how the models are built and what they learned from, which is why it cannot be fixed by good intentions at the point of use.

The Gap between belief and concern

The HR training article reports a contradiction worth reading slowly. Half of companies cite AI introducing bias as a top concern in hiring, and 57% cite AI screening out qualified candidates as a separate top concern. Set against that, 80% of HR professionals still believe AI reduces unconscious bias at the early screening stage.

Both things are true at once inside the same organisations. The article’s point is that a training session which teaches the screening tool without naming that contradiction leaves the belief in place and the concern unaddressed. It is not that either group is wrong. A tool can reduce one kind of unfairness and introduce another, and a team that has only heard the first half will not look for the second.

The article’s defensible position is specific. Filtering candidates against published criteria is an arguable use of the technology. Ranking them by an opaque score is not. A curriculum has to leave people able to tell the two apart, and not treat “the tool does bias-aware screening” as the end of the conversation.

What the screening slide lists

The HR course the article draws on includes a slide titled “CV Screening: Risks and solutions”. Its risks are everyone qualified, invented preferences, no names is not enough, no explanation, ranking is profiling and confidence. Its five fixes are check don’t rank, quote the line, finding not judging, name a published criterion for every exclusion, and anonymise locally.

Read together, the fixes describe a method. The tool is used to find things in a CV and quote the line it found, which a person can verify. It is not used to judge a person or to produce a score. Every exclusion has to point to a criterion that was published in advance. The data is anonymised before the tool sees it, and locally, so the anonymising does not itself send the data somewhere new.

The HR course teaches this on real material. Its ethics section covers handling people and not documents, with bias, transparency and over-reliance and real cases. Its screening section covers the risks, the fixes and the honest answer. Its stated outcome is that participants anonymise data locally and hold a defensible position on CV screening.

What the EU AI Act says about Recruitment

The EU AI Act classifies systems used for recruitment and candidate evaluation as high-risk. The high-risk AI system page lists what that covers: job-advert targeting, CV filtering and candidate scoring. It also covers systems that decide promotion, termination or task allocation, and performance monitoring once someone is employed.

The classification puts a documentation and risk-management obligation on the system, whichever supplier or platform runs it. The page’s illustration, labelled there as not a real client, is a small company with two AI uses. One drafts internal meeting summaries and sits outside the category. The other reads incoming CVs and produces a ranked shortlist, which is a recruitment use and so falls inside it. Both might run in the same tool. The use sets the class, not the tool.

The same HR data is personal data under GDPR. The HR article argues that a curriculum which treats the two as separate topics describes half the picture. This page is not legal advice, and no audit substitutes for advice from a qualified lawyer.

The Provider and the Deployer

A common mistake is to assume the vendor carries all of it. The HR training article describes the split. Most of the Act’s obligation is heavier on the provider that built the system: a risk management system across the tool’s lifecycle, training data that is representative and with as few errors as can be managed, logging, and instructions that let the people running it apply oversight properly.

A deployer is the HR team actually running the system day to day. The article says a deployer carries fewer duties than the provider, but fewer is not the same as none. “The vendor handles compliance” is not itself a defensible answer, and a training session should not let a team walk out believing it. The line between provider and deployer is part of the same session as the bias material, not a separate legal briefing.

Keeping a person in the Decision

The site’s position on who decides is consistent across its pages. The CV page says AI can filter out unqualified applicants but should never make the final pick. The delegation ladder puts hiring at Never delegate, and the task and decision article explains why. The human in the loop page reports that AI reaches about 75 to 80% accuracy on candidate shortlisting against 85 to 90% for experienced humans, close enough to support the call and not replace it.

The AI governance page says a hiring decision needs a named human checkpoint, and that the checkpoint has to be staffed. A rule that says “a person should check” is a hope. A control says who checks, at what stage and against what standard.

The CV page also notes a simpler route: if the criteria are fully known, a form or code applies them correctly every time. In that case AI is better used to build the system than to be the decision maker.

Bias and Hallucination

The two are often treated as one problem. They are separate failures. A hallucination is a fluent, confident statement that is simply invented. Bias is an answer that skews because of the material the model was trained on. A biased answer can contain no invented facts at all and still be a problem. A hallucinated answer can carry no bias at all and still be wrong.

Both need a human check, but not the same one. Catching a hallucination means comparing the claim with what is actually in the source. Catching bias means asking whether the answer would have been the same for a different person, group or viewpoint. A reviewer who only does the first will miss the second.

Why Tool Training alone does not close the Gap

The HR article’s central argument is that most AI training built for HR teams teaches the tools and stops there. It leaves out the subjects that cause damage when they are missing: bias in AI-assisted decisions, and the compliance obligations specific to HR data.

Its figures show why that is risky. 49% of organisations have a policy governing staff AI use, and only a quarter of those believe it is future-proof. 57% of HR professionals working under a state-level workforce AI regulation do not know it applies to them, and of those who do know, only 12% have taken a step to comply. A tool demonstration cannot close gaps like those, because they are not about the tool.

The article also notes a pressure from outside the HR team’s own AI use: 67% of HR leaders say reviewing AI-generated job applications has slowed hiring, and 84% report a heavier workload. That is part of why a team needs a repeatable method and not an improvised one.

What a curriculum has to name

The article’s recommendation is a programme that treats bias and compliance as named sessions, not a caveat slide before the demo. That means a defensible answer on screening, a data protection session covering anonymisation and local hosting, and the exact point at which a person, not the model, makes the final call. It also says the AI literacy of HR comes first, because HR is generally expected to lead upskilling for the rest of the organisation.

Whether it worked is a separate question from whether it happened. The article’s test is behavioural: can the people who sat through it explain, in their own words and without notes, why a given AI-assisted decision is or is not defensible, and point to the specific safeguard that makes it so. Attendance and a certificate say nothing about that.

What to do first

Start with one question about any AI use that touches people: could it screen someone out? If it could, name the published criterion for every exclusion, keep a named person on the final call, and anonymise the data before a tool sees it. Then check whether bias and the Act are named sessions in whatever training the team is getting, or an assumed footnote.

An audit assesses the Act at the Identify step, inside the ranking of opportunities and not as a review added at the end, so a hiring use is classified before anything is built. See AI training for HR teams: what should it cover for the full argument and the AI essentials for human resources course for how it is taught.