Quality of Online Password Checkers

There are many online password checkers on the Internet. In this post, we evaluate 24 of them that are somehow serious. We expect all these checkers to work well for common trivial cases – i.e. to recognize a very simple weak password as a weak password and a very complex strong password as a strong password. But, just to be sure, we do test that feature too. Our primary focus here, however, is on passwords that are interesting somehow. The main goal is to see how many of the tested password checkers can evaluate strength of passwords correctly.

Smarter Attacks = Better Checkers

Now, what does it mean to evaluate a strength of a password correctly? It should be noted that from the 24 checkers we compare here, few of them actually support some kind of a dictionary attack. This is why we focus solely on brute-force attacks, which is what all the tested checkers support. Password checkers themselves do not really perform brute-force attacks, they just try to count or estimate how long would it take to an attacker to perform a successful brute-force attack on the given password. We think that the smarter attacker the password checker can “simulate”, the better the password checker is. For example, assume we have this very simple password “qregjperb”. It is simple because it uses lower case Latin alphabet characters only – i.e. set [a-z]. Since it is known that many computer users do use weak passwords, a smart attacker might try to crack this password assuming the limited character set. In this case of 8 characters long password, the attacker would end with approximately 26⁸ possible combinations (we omit the need to test shorter passwords first because the actual length is unknown to the attacker). On the other hand, not that smart attacker might take, for example, a whole ASCII set of characters (256 characters) and thus face a much larger task of approximately 256⁸ possible combinations. It is thus obvious that a password checker, which does not consider these “smart” technique of an attacker, tends to claim many passwords as strong while in fact they can be cracked pretty fast.

This simple example gives us a clue on how to answer the question of correct password strength evaluation. A password should be considered weak if a reasonably smart attacker can crack it quickly. A strong password can not be cracked using a brute-force attack quickly even if the attacker is smart. Now, what does it mean to be a smart brute-force attacker? The answer is a good set of rules that will help to crack most of the passwords more quickly while not penalizing the rest of the passwords that much. This means that if the assumed attacker gets an access to a large database of somehow hashed or encrypted passwords (e.g. from LinkedIn), it is his intention to crack as many of them as quickly as possible. A well designed cracking rule set (which would of course include a dictionary attack too, but let’s focus on the brute-force technique here) is the key to success here. Having a specific set of rules, we can say that every password has its well defined complexity. We can compare complexity of every two passwords and determine, which is harder to crack. A poorly built password checker calculating only with a basic set of cracking rules may then evaluate a weaker password to be stronger than a stronger one and vice versa. A good password checker should not produce such a mistake.

Test Cases

Here are 8 password samples that we have used to test online password checkers.

1. The first test is just to verify that the checker works on trivial cases. There are two passwords in this tests:
   

   K$a4j_GJ9.Bzfj3Ajkw7

   and

   abcd

   The first one is obviously a very secure password and the other one is a very bad password. A password checker passes this test if it rates the first password as secure and the second as insecure.

2. This test checks whether the password checkers do not blindly rely on the length of the given password. The sample password for this tests is
   

   pppppppppppppppppppppppppppppppppppppppp

   which is simply a password consisting of 40 same characters. Such a password is definitely crackable in very short time if the attacker uses a smart rule. This password should be evaluated as a weak password.

3. In contrast with the previous test, this one checks whether the checker does not incorrectly rely on using various character sets or a non-trivial number of characters from one character set. The sample password for this tests is
   

   abccabcbacbbabcbabacbcacccccbacbbacbabca

   This password is quite strong despite the fact that it is built only from three different characters. Due to its length the number of possible combinations is too big that even if the attacker was aware of the fact that only ‘a’, ‘b’, and ‘c’ characters were used, it would be impossible for the attacker to crack it. Note that with the length of 40 characters, these statements are valid when three or more different characters are used. Using just two characters (or one) might not be enough if the attacker has significant computational power available. A password checker passes this test if it rates the password with one of its higher ranks.

4. This is the third test to check whether online checkers do not penalize too much for not using more than a single common set of characters. The sample password for this tests is
   

   abcbaudhuiahafjkghauirnvaruianeruiaepger

   This password is very strong against brute-force attacks mostly due to its length. A password checker passes this test if it rates the password with one of its higher ranks.

5. In this test we want password checkers to recognize the following password as not very strong. The sample password for this tests is
   

   abccabcbacbbabcbabac

   Despite its length, which would be sufficient to create quite strong passwords if more characters were used, this particular password is not that strong because only three different characters are used. Using a smart set of rules an attacker might crack this password in very short time. An online password checker passes this test in case this password does not reach any of the higher ranks the checker provides.

6. This test checks whether sizes of common character sets are evaluated properly. Sample passwords for this tests are
   

   goorsoprfaQ

   and

   goorsoprfa0

   The first password uses lower and upper case characters of Latin alphabet, the second password uses lower case characters of Latin alphabet and a digit character. The first password is slightly stronger, but it is enough for us if checkers evaluate them to be of the same strength or that the first one is stronger.

7. UTF-8 test. This test is done only for checkers that use UTF-8 as a default encoding. It verifies that the checker is able to handle unicode characters correctly. Sample passwords for this tests are
   

   goorsoprfa黑

   and

   goorsoprfa!

   Considering sizes of both used character sets, the first password should be evaluated to be as strong as or stronger than the second one. In fact, the last character of the first password is from a much larger set than the last character of the second password, which makes the first password stronger against brute-force attacks. We consider the test to be passed even if both passwords are considered of the same strength because some checkers might not provide enough granularity to distinguish between these two passwords.

8. Second UTF-8 test, much like the previous one. Also this one is done only for checkers that use UTF-8 as a default encoding. Sample passwords for this tests are
   

   今天黑色星期五百科

   and

   AQ_1c.8aZ

   This case is a little bit more tricky than the previous one because here we have the whole password formed of Chinese symbols, which might be incorrectly interpreted as a less secure password, from the brute-force attack point of view, than the combination of the Latin lower case alphabet, the Latin upper case alphabet, digits and common ASCII symbols. However, it is still the first password that is more secure even if consider only the size of used character sets. Passing this tests requires the checker to evaluate the first password to be as strong as the second one or stronger.

Results

^† Please note that it is not recommended to use GeodSoft Password Evaluator/Checker because it requires the password to be sent in an unencrypted form over the network.

Summary

Fortunately, all the tested online password checkers passed test #1. Not passing this one would be very bad. The test case 8 seemed to be the hardest one, only few password checkers passed it. Tests #3 and #4 were also quite hard.

Only four checkers failed no more than 1 test and one of those four should not be used because of its insecure implementation. Except for our Password Checker Online, which is the only one that passed all of the tests (but that is not very surprising since we designed the tests, right?), we can recommend you to use BA.net Password Security Checker. This tool failed only the test #8, which means that unless you use characters outside the ASCII encoding in your passwords, you should be fine with it. zxcvbn might also be a good choice.