This PIN Can Be Easily Guessed


Philipp Markert, Daniel V. Bailey, Maximilian Golla, Markus Dürmuth, and Adam J. Aviv
Ruhr University Bochum & Max Planck Institute for Security and Privacy & The George Washington University
ACM Transactions on Privacy and Security and IEEE Symposium on Security and Privacy

tl;dr: Study of user-chosen 4- and 6-digit PINs collected on smartphones for device unlocking. Measuring the effects of blocklists, where a set of "easy to guess" PINs is disallowed during selection.

Overview



PIN Selection on Smartphones

We conducted a user study focused on the selection of Personal Identification Numbers (PINs) based on data collected from users specifically primed for the smartphone unlock setting. Despite the rise of biometrics, such as fingerprint or facial recognition, devices still require PINs, e.g., after a restart or when the biometric fails.

As a mechanism for improving PIN selection, we also studied how PINs are affected by blocklisting. A blocklist is a set of "easy to guess" PINs, which triggers a warning to the user. Apple iOS devices show the warning "This PIN Can Be Easily Guessed" with a choice to "Use Anyway" or "Change PIN."

Responsive image

Responsive image

Findings

Our study found there is little benefit to longer 6-digit PINs as compared to 4-digit PINs. Our participants tended to select more-easily guessed 6-digit PINs when considering the first 40 guesses of an attacker. Moreover, our results show that currently employed PIN blocklists are ineffective. Through quantitative and qualitative feedback, we found that participants perceive that blocklisting will improve their PINs without impacting usability.

# Strategy Description Example
1 Dates Special date like birthday, anniversary, or graduation day 1987 / 112518
2 Memorable Memorability was the main concern 2827 / 777888
3 Pattern Visualized a pattern on the PIN pad 2580 / 137955

Apple's iOS Passcode Blocklist

The iOS blocklist of passcodes was obtained via brute-force. To test if a PIN is blocklisted, one only needs to try it and see if a warning appears. During the initial iOS setup there is no rate-limiting in place, which enabled us to quickly test all possible PINs.

We constructed a device to automate this process using a Raspberry Pi equipped with a camera. The device emulates a USB keyboard, which is connected to the iPhone. After entering a PIN, the camera takes a photo of the iPhone screen. The photo is sent to a remote server, where the presence of the blocklist warning is detected by extracting the text in the photo.

More details and photos of the PIN extraction device can be found here: Extracting iOS' Passcode Blocklist

Responsive image

Datasets



Blocklists

In our study, we tested various blocklists in enforcing and non-enforcing settings. Besides Apple's 4-digit and 6-digit blocklists, we also created data-driven blocklists that are significantly (10x) smaller (27/29 PINs) and (10x) larger (2740/291,000 PINs) than the iOS 4/6-digit blocklists. To foster future research on this topic, we publish some of the studied blocklists here.

Apple's iOS Passcode blocklist was last updated with iOS 10.3 (14E277). Our data-driven blocklists, are based on PINs released by Daniel Amitay in 2011 and are partly reordered using a Markov model.
# Name Source Length Blocklisted Download
1 iOS-4-digit Apple iOS 4-digit 274
2 iOS-6-digit Apple iOS 6-digit 2,910
3 DD-4-digit-27 Top Amitay 4-digit 27
4 DD-4-digit-2740 Top Amitay 4-digit 2,740
5 DD-6-digit-29 Top RockYou 6-digit 29
6 DD-6-digit-291000 Top RockYou 6-digit 291,000

PIN Datasets

Before our user study, the most realistic set of 4-digit PINs was from 2011, where Daniel Amitay developed the iOS application "Big Brother Camera Security." The app mimicked a lock screen allowing users to set a 4-digit PIN. Amitay anonymously and surreptitiously collected 4-digit PINs (204,432). As there was no similar 6-digit PIN data available to inform our attacker, we relied on 6-digit PINs extracted from the RockYou password leak, similar to Bonneau et al. (2012) and Wang et al. (2017). PINs are extracted from consecutive sequences of exactly n-digits in leaked password data. By following this method, we extracted 6-digit PINs from the RockYou password leak, which we refer to as RockYou-6-digit (2,758,490 PINs). For comparison, we also provide a 4-digit version of the RockYou dataset (1,780,587 PINs).

Our attacker guesses PINs in decreasing probability order based on the Amitay-4-digit and RockYou-6-digit datasets. When two or more PINs share the same frequency, i.e., it is not possible to directly determine a guessing order, we ordered those PINs using a Markov model.
# Name Source Length PINs Download
1 Amitay-4-digit Amitay 4-digit 204,432
2 RockYou-4-digit RockYou 4-digit 1,780,587
3 RockYou-6-digit RockYou 6-digit 2,758,490

User-Chosen PINs

We conducted a user study of 4- and 6-digit PINs using Amazon Mechanical Turk (MTurk) with (n=1,705) participants over a period of three weeks. To mimic the PIN creation process in our browser-based study, participants were restricted to mobile devices by checking the User-Agent string. We applied a 12-treatment, between-subjects study protocol for the PIN selection criteria, e.g., 4- vs. 6-digit with or without blocklisting. At the end of the study, we collected 851 and 854 PINs, 4- and 6-digits, respectively. These PINs were all selected, confirmed, and recalled. We additionally recorded all intermediate PIN selections, such as what would happen if a selected PIN was not blocklisted and the participant did not have to select a different PIN.

We share this dataset of user-chosen PINs with other research institutions upon request. Contact details () are published along with the papers.


Technical Papers


An extended version of our work appeared in the ACM Transactions on Privacy and Security (ACM TOPS '21, Vol. 24, No. 4).


Our work appeared at the 41st IEEE Symposium on Security and Privacy (IEEE SP '20).


Abstract

In this article, we provide the first comprehensive study of user-chosen four- and six-digit PINs (n=1,705) collected on smartphones with participants being explicitly primed for device unlocking. We find that against a throttled attacker (with 10, 30, or 100 guesses, matching the smartphone unlock setting), using six-digit PINs instead of four-digit PINs provides little to no increase in security and surprisingly may even decrease security. We also study the effects of blocklists, where a set of "easy to guess" PINs is disallowed during selection. Two such blocklists are in use today by iOS, for four digits (274 PINs) as well as six digits (2,910 PINs). We extracted both blocklists and compared them with six other blocklists, three for each PIN length. In each case, we had a small (four-digit: 27 PINs; six-digit: 29 PINs), a large (four-digit: 2,740 PINs; six-digit: 291,000 PINs), and a placebo blocklist that always excluded the first-choice PIN.

For four-digit PINs, we find that the relatively small blocklist in use today by iOS offers little to no benefit against a throttled guessing attack. Security gains are only observed when the blocklist is much larger. In the six-digit case, we were able to reach a similar security level with a smaller blocklist. As the user frustration increases with the blocklists size, developers should employ a blocklist that is as small as possible while ensuring the desired security. Based on our analysis, we recommend that for four-digit PINs a blocklist should contain the 1,000 most popular PINs to provide the best balance between usability and security and for six-digit PINs the 2,000 most popular PINs should be blocked.


BibTeX

If you want to cite the dataset or the extended paper, please use the following BibTeX entry:

@article{markert-21-unlock-pins,
    author = {Markert, Philipp and Bailey, Daniel V. and Golla, Maximilian and D\"{u}rmuth, Markus and Aviv, Adam J.},
    title = {{On the Security of Smartphone Unlock PINs}},
    journal = {ACM Transactions on Privacy and Security},
    year = {2021},
    volume = {24},
    number = {4},
    pages = {30:1--30:36},
    month = nov,
    publisher = {ACM}
}