Cryptography is crucial for data security in today's software systems. That's why developers are turning to static analysis tools, known as crypto-detectors, to spot and fix problems with crypto-API use. But do these tools really work? To answer this, researchers created the MASC framework. This framework uses mutation testing to systematically evaluate crypto-detectors.
First, they made a list of $105$ real-life misuse cases, grouped into nine categories. Then, they developed $12$ special tools to create tons of variations of these cases for thorough testing. Using MASC, they tested nine major crypto-detectors and found $19$ serious flaws that were never documented before. These flaws make it hard for crypto-detectors to find misuses in real software.
This discovery raises important questions about how crypto-detectors are made. It also points to how we can design better ones that really focus on security.
