💡 Intersecting topics with Machine Learning Security

[shsingh]

When discussing Machine Learning Security it becomes evident that there are intersecting topics to consider:

• Privacy
• Ethics

These topics are quite broad and cover things outside security, nonetheless there are intersections. This means that the security of machine learning systems can have an effect on privacy and ethical related issues.

❓ ## Questions for discussion

• What do others think?
• Is there other other topics that intersect like the ones above?
• Should there be a way for the MLSEC TOP 10 mentioning or considering these? Or should the MLSEC TOP 10 be analogous to other Top 10 lists and cater for developers and application security practitioners only?

[mik0w]

Is there other other topics that intersect like the ones above?
I think that one of the topics that intersect is ML safety. Security and safety seem to cover similar areas, but many sources distinguish between these fields

[edthedev]

Two challenge areas that jump out to me are equity (from systemic bias) and accessibility (from new implementations that don't understand many users' needs). I've found that most new technology comes with new equity and accessibility challenges, particularly during early learning and adoption phases of the technology lifecycle.

I do think that 'Ethics' is a solid umbrella to discuss equity and accessibility under, so I'm satisfied with the two high level topics above.

If the sub-topics under 'Ethics' gets too long, we can always revisit.

[robvanderveer]

Privacy is relevant to security where privacy is about information security. The other elements are best left out of scope of an ML sec top 10: use limitation, purpose specification, fairness, transparency, privacy rights, data accuracy and consent.
There are ethical aspects of AI. that are largely not part of privacy, and those are also best left out of the top 10: accountability and liability, societal impact, environmental impact. Some privacy frameworks exclude fairness, bias, transparency and explainability, but that's an increasingly unpopular view.
I regard data minimization and storage limitation as security controls by the way.

Safety and security are also different things. Safety is more tied to correctness, reliability and robustness - where robustness is partly connected to security. After all, a security threat can mean a safety risk.
For the top 10, I believe there are no issues that are more safety than they are security.

For more details, see https://owasp.org/www-project-ai-security-and-privacy-guide/

[mik0w]

For the top 10, I believe there are no issues that are more safety than they are security.

I'd consider ML01 - adversarial attacks a safety issue (although it's a mix of both). When I think of adversarial ML, those example of making autonomous vehicles think that "stop" sign is "60 mph" sign always come to my mind :) https://arxiv.org/pdf/2101.06092.pdf

[robvanderveer]

Adversarial attacks are a security issue because an attacker is trying to change the behavior of a system that then may affect safety. Similarly, denial-of-service of any system is a security issue as a cause, and sometimes a safety issue as an effect. In the same fashion, a data leak can be a business continuity risk. When discussing technical security threats, effects such as safety and business continuity are an interesting dimension, but they do not shape the discussion of the threats.

[edthedev]

When we use a Top Ten list to drive analysis, my team discusses the impact of the top ten threats to the accessibility, equity, privacy, reliability and safety of the system being developed.

That said, I'm not sure those topics can be covered in absence of the system under scrutiny. A traffic light controller uses machine learning differently than a job application website.

So I'm thinking that we should try to ensure that accessibility, equity, privacy, reliability and life-safety are included in any examples we provide to help developers understand the nature of the threats - rather than trying to tackle them in the threat overviews or descriptions.