When algorithms make decisions about people

Automated and semi-automated systems now touch candidate sourcing and screening, internal mobility, performance signals, workforce planning, engagement analysis, scheduling and attrition prediction. Very few of them make a final decision alone. Most shape the option set a human sees — which is a quieter and more consequential form of influence, because it leaves no trace of what was removed.

The most widely cited cautionary example remains Amazon's experimental recruiting tool, which the company reportedly abandoned after finding it disadvantaged applications associated with women; it was never rolled out (Reuters, 2018). The instructive part is not that a model was biased, but that the bias came from learning faithfully from historical hiring data.

Pattern detection: the system has found a statistical regularity. This is a description of data, not of the world. It may reflect the phenomenon, or the sampling, or the labelling, or the organisation's past behaviour.

Correlation: two variables move together. Correlation supports a question. It does not support a conclusion about mechanism, and it is routinely confounded by variables nobody recorded.

Possible contributing factor: an explanation consistent with the pattern, held provisionally, alongside the alternatives that are also consistent with it.

Causation: a claim that changing one thing changes the other. This requires evidence a typical people-analytics dataset cannot supply — controlled comparison, temporal ordering, or a well-founded mechanism, and the exclusion of plausible confounders.

Legal discrimination: a legal determination made against a jurisdiction's statutory framework, its defined protected characteristics, its concepts of direct and indirect treatment, and any objective justification. A model output is evidence that may inform such a determination. It is not the determination.

Collapsing these five into one another is the most common analytical failure in this field, and it fails in both directions: organisations either over-claim what the data proves, or dismiss a genuine signal because it is not yet proof.

Models learn from records of past organisational behaviour. Where performance ratings encoded manager leniency, where progression reflected access to sponsorship, where job descriptions were written for the person who last held the role, the historical data is a record of the organisation's habits — including the ones it is trying to change.

Proxy variables make this harder to see. Postcode, education route, career gaps, tenure patterns and even communication style can carry information about protected characteristics without naming them. Removing the protected field does not remove the signal.

The UK Information Commissioner's Office sets out expectations for fairness, transparency, lawful basis and accountability where AI processes personal data, including the need to be able to explain decisions to the people affected (ICO, 2023). The EU AI Act, adopted in 2024, classifies a range of employment and worker-management uses as high risk and attaches obligations including risk management, data governance, human oversight, logging and transparency (Regulation (EU) 2024/1689).

New York City's Local Law 144 requires bias audits and candidate notification for certain automated employment decision tools, an early example of jurisdiction-specific obligations that multinational employers now have to reconcile across markets.

Human oversight only counts when it is real. A reviewer who lacks the time, the information or the standing to overturn a system's recommendation provides governance on paper and rubber-stamping in practice.

Legal exposure is the risk most often discussed and rarely the first to materialise. Before that come employee-relations consequences when people conclude that decisions about them are made by systems nobody can explain; reputational consequences when that conclusion becomes public; and commercial consequences when an organisation systematically overlooks capability it already employs.

Trust is the operative asset. Once employees believe the process is opaque, every subsequent decision — including the fair ones — is interpreted through that belief.

Maintain an inventory of systems that influence people decisions, including the ones procured inside a wider platform. Record purpose, data sources, model outputs and the human decision each one feeds. Test outcomes across groups regularly rather than once at procurement. Document the alternative explanations considered for any pattern found. Keep escalation routes that a human can actually use. And be explicit with employees about where these systems are in play.

None of this requires an organisation to reject the technology. It requires the organisation to remain the decision-maker.