This PIN Can Be Easily Guessed
Philipp Markert, Daniel V. Bailey, Maximilian Golla, Markus Dürmuth, and Adam J. Aviv
Ruhr University Bochum & Max Planck Institute for Security and Privacy & The George Washington University
ACM Transactions on Privacy and Security and IEEE Symposium on Security and Privacy
tl;dr: Study of user-chosen 4- and 6-digit PINs collected on smartphones for device unlocking. Measuring the effects of blocklists, where a set of "easy to guess" PINs is disallowed during selection.
Overview
PIN Selection on Smartphones
We conducted a user study focused on the selection of Personal Identification Numbers (PINs) based on data collected from users specifically primed for the smartphone unlock setting. Despite the rise of biometrics, such as fingerprint or facial recognition, devices still require PINs, e.g., after a restart or when the biometric fails.
As a mechanism for improving PIN selection, we also studied how PINs are affected by blocklisting. A blocklist is a set of "easy to guess" PINs, which triggers a warning to the user. Apple iOS devices show the warning "This PIN Can Be Easily Guessed" with a choice to "Use Anyway" or "Change PIN."
Findings
Our study found there is little benefit to longer 6-digit PINs as compared to 4-digit PINs. Our participants tended to select more-easily guessed 6-digit PINs when considering the first 40 guesses of an attacker. Moreover, our results show that currently employed PIN blocklists are ineffective. Through quantitative and qualitative feedback, we found that participants perceive that blocklisting will improve their PINs without impacting usability.
| # | Strategy | Description | Example |
|---|---|---|---|
| 1 | Dates | Special date like birthday, anniversary, or graduation day | 1987 / 112518 |
| 2 | Memorable | Memorability was the main concern | 2827 / 777888 |
| 3 | Pattern | Visualized a pattern on the PIN pad | 2580 / 137955 |
Apple's iOS Passcode Blocklist
The iOS blocklist of passcodes was obtained via brute-force. To test if a PIN is blocklisted, one only needs to try it and see if a warning appears. During the initial iOS setup there is no rate-limiting in place, which enabled us to quickly test all possible PINs.
We constructed a device to automate this process using a Raspberry Pi equipped with a camera. The device emulates a USB keyboard, which is connected to the iPhone. After entering a PIN, the camera takes a photo of the iPhone screen. The photo is sent to a remote server, where the presence of the blocklist warning is detected by extracting the text in the photo.
More details and photos of the PIN extraction device can be found here: Extracting iOS' Passcode Blocklist
Datasets
Blocklists
In our study, we tested various blocklists in enforcing and non-enforcing settings. Besides Apple's 4-digit and 6-digit blocklists, we also created data-driven blocklists that are significantly (10x) smaller (27/29 PINs) and (10x) larger (2740/291,000 PINs) than the iOS 4/6-digit blocklists. To foster future research on this topic, we publish some of the studied blocklists here.
| # | Name | Source | Length | Blocklisted | Download |
|---|---|---|---|---|---|
| 1 | iOS-4-digit | Apple iOS | 4-digit | 274 | |
| 2 | iOS-6-digit | Apple iOS | 6-digit | 2,910 | |
| 3 | DD-4-digit-27 | Top Amitay | 4-digit | 27 | |
| 4 | DD-4-digit-2740 | Top Amitay | 4-digit | 2,740 | |
| 5 | DD-6-digit-29 | Top RockYou | 6-digit | 29 | |
| 6 | DD-6-digit-291000 | Top RockYou | 6-digit | 291,000 |
PIN Datasets
Before our user study, the most realistic set of 4-digit PINs was from 2011, where Daniel Amitay developed the iOS application "Big Brother Camera Security." The app mimicked a lock screen allowing users to set a 4-digit PIN. Amitay anonymously and surreptitiously collected 4-digit PINs (204,432). As there was no similar 6-digit PIN data available to inform our attacker, we relied on 6-digit PINs extracted from the RockYou password leak, similar to Bonneau et al. (2012) and Wang et al. (2017). PINs are extracted from consecutive sequences of exactly n-digits in leaked password data. By following this method, we extracted 6-digit PINs from the RockYou password leak, which we refer to as RockYou-6-digit (2,758,490 PINs). For comparison, we also provide a 4-digit version of the RockYou dataset (1,780,587 PINs).
User-Chosen PINs
We conducted a user study of 4- and 6-digit PINs using Amazon Mechanical Turk (MTurk) with (n=1,705) participants over a period of three weeks. To mimic the PIN creation process in our browser-based study, participants were restricted to mobile devices by checking the User-Agent string. We applied a 12-treatment, between-subjects study protocol for the PIN selection criteria, e.g., 4- vs. 6-digit with or without blocklisting. At the end of the study, we collected 851 and 854 PINs, 4- and 6-digits, respectively. These PINs were all selected, confirmed, and recalled. We additionally recorded all intermediate PIN selections, such as what would happen if a selected PIN was not blocklisted and the participant did not have to select a different PIN.
We share this dataset of user-chosen PINs with other research institutions upon request. Contact details () are published along with the papers.
Technical Papers
An extended version of our work appeared in the ACM Transactions on Privacy and Security (ACM TOPS '21, Vol. 24, No. 4).
- Extended Paper Includes new data about 6-digit PINs and an extended analysis.
Our work appeared at the 41st IEEE Symposium on Security and Privacy (IEEE SP '20).
Abstract
In this article, we provide the first comprehensive study of user-chosen four- and six-digit PINs (n=1,705) collected on smartphones with participants being explicitly primed for device unlocking. We find that against a throttled attacker (with 10, 30, or 100 guesses, matching the smartphone unlock setting), using six-digit PINs instead of four-digit PINs provides little to no increase in security and surprisingly may even decrease security. We also study the effects of blocklists, where a set of "easy to guess" PINs is disallowed during selection. Two such blocklists are in use today by iOS, for four digits (274 PINs) as well as six digits (2,910 PINs). We extracted both blocklists and compared them with six other blocklists, three for each PIN length. In each case, we had a small (four-digit: 27 PINs; six-digit: 29 PINs), a large (four-digit: 2,740 PINs; six-digit: 291,000 PINs), and a placebo blocklist that always excluded the first-choice PIN.
For four-digit PINs, we find that the relatively small blocklist in use today by iOS offers little to no benefit against a throttled guessing attack. Security gains are only observed when the blocklist is much larger. In the six-digit case, we were able to reach a similar security level with a smaller blocklist. As the user frustration increases with the blocklists size, developers should employ a blocklist that is as small as possible while ensuring the desired security. Based on our analysis, we recommend that for four-digit PINs a blocklist should contain the 1,000 most popular PINs to provide the best balance between usability and security and for six-digit PINs the 2,000 most popular PINs should be blocked.
BibTeX
@article{markert-21-unlock-pins,
author = {Markert, Philipp and Bailey, Daniel V. and Golla, Maximilian and D\"{u}rmuth, Markus and Aviv, Adam J.},
title = {{On the Security of Smartphone Unlock PINs}},
journal = {ACM Transactions on Privacy and Security},
year = {2021},
volume = {24},
number = {4},
pages = {30:1--30:36},
month = nov,
publisher = {ACM}
}