IBM to alert companies if their AI systems risk becoming racist or sexist

IBM is to tell people using its artificial intelligence software if their system is biased. 

The company will automatically detect whether the data used by AI systems includes characteristics such as age, sex, ethnicity and marital status, and recommend that these are monitored to make sure people are not discriminated against by the software. 

AI software used to rank job applicants or predict criminal behaviour has been found to be inadvertently biased based on race or gender.

The software, Watson OpenScale, will now also recommend which attribute should be used as reference for others to be measured against. 

For example, within the "sex" category it will suggest that "women" and "non-binary" are monitored against "men", to make sure all three groups are being treated equally. 

Previously, users had to manually select aspects to monitor.  Last year it emerged that Amazon had abandoned development of an AI hiring tool after it penalised women. 

The system had been trained on CVs submitted to the company, most of which came from men, so it "learned" that successful candidates were more likely to be male and downgraded potential employees who included words like "women's" in their application.

Facial recognition systems have also been criticised for being better at identifying and analysing white, male faces than women and non-white people.