Android - Require Fingerprint Authentication To Access (rsa/pss) Signing Key

I'm currently creating a form of challenge-response authentication for a project I need for my Master thesis in computer science. For this purpose, I need to create an RSA-PSS sign

Solution 1:

Change your getPrivateKey method to:

private PrivateKey getPrivateKey() {

     KeyStore store = KeyStore.getInstance("AndroidKeyStore");
     store.load(null);

     return (PrivateKey) keyStore.getKey("authKey", null));
 }

In your code, you iterate through all keys and grep the last one, which isn't necessarily the one that you want - or even worse: return null if that key doesn't have a private key...

If you want to check whether a key exist:

 if (store.containsAlias(keyName)) {
     ...
 }

Solution 2:

I finally found a solution.

There were actually two problems at hand here.

1. I tried to use SHA-512 as a mask-generation function for RSA/PSS, which is "probably" unsupported by the cryptographic library that Android uses.

2. I tried to sign an empty (0-byte) message, which somehow appears to be "problematic".

When I both changed the MGF to SHA-256and made the message 64 bytes long, signature generation succeeded.

Now, both "requirements" appear to be a bit "weird".

First, you can indeed use SHA-512 as an MGF for RSA/PSS, as long as you setUserAuthenticationRequired(false), so it has to be supported by the cryptographic library. It's only when you enable authentication that it suddenly fails and you have to fall back to SHA-256. I did not perform extensive testing which hash functions work as MGFs for RSA/PSS with authentication and which do not. I just found that SHA-512 does not work but SHA-256 does, so the choice of MGF is somehow "restricted" when authentication is enabled.

Second, your message needs to have a certain minimal size in order for it to be signed with authentication enabled. For example, you cannot sign an empty buffer. This makes no sense to me at all since the first step in RSA/PSS is to apply a cryptographic hash function to the message, the output of which is fixed length, so the signature scheme really shouldn't care how long or short the message is, but apparently it does. Like before, I didn't perform extensive testing to find the exact cutoff point where the message becomes "long enough" for signing. However, I found that a 64 byte message can be signed, while an empty (0 byte) message cannot, so the minimal length is somewhere within [1; 64] bytes, both limits inclusive.

Note that, as of now, this seems to be documented nowhere and also the exception thrown is of no use. It just says "signature verification failed" (yes, it says "verification" even though we're actually generating a signature), so you have no idea that you have to change the MGF and the length of the message to be signed.

Due to this, there might be more to it that I haven't found. I just found this parametrization by "trial and error" and thus have no idea what the actual constraints of the cryptographic library look like.