This PIN Can Be Easily Guessed


Philipp Markert, Daniel V. Bailey, Maximilian Golla, Markus Dürmuth, and Adam J. Aviv
Ruhr University Bochum & Max Planck Institute for Security and Privacy & The George Washington University
IEEE Symposium on Security and Privacy, San Francisco, California, USA, May 2020

tl;dr: Study of user-chosen 4- and 6-digit PINs collected on smartphones for device unlocking. Measuring the effects of blacklists, where a set of "easy to guess" PINs is disallowed during selection.

Overview



PIN Selection on Smartphones

We conducted a user study focused on the selection of Personal Identification Numbers (PINs) based on data collected from users specifically primed for the smartphone unlock setting. Despite the rise of biometrics, such as fingerprint or facial recognition, devices still require PINs, e.g., after a restart or when the biometric fails.

As a mechanism for improving PIN selection, we also studied how PINs are affected by blacklisting. A blacklist is a set of "easy to guess" PINs, which triggers a warning to the user. Apple iOS devices show the warning "This PIN Can Be Easily Guessed" with a choice to "Use Anyway" or "Change PIN."

Responsive image

Responsive image

Findings

Our study found there is little benefit to longer 6-digit PINs as compared to 4-digit PINs. Our participants tended to select more-easily guessed 6-digit PINs when considering the first 40 guesses of an attacker. Moreover, our results show that currently employed PIN blacklists are ineffective. Through quantitative and qualitative feedback, we found that participants perceive that blacklisting will improve their PINs without impacting usability.

# Strategy Description Example
1 Dates Special date like birthday, anniversary, or graduation day 1987 / 112518
2 Memorable Memorability was the main concern 2827 / 777888
3 Pattern Visualized a pattern on the PIN pad 2580 / 137955

Apple's iOS Passcode Blacklist

The iOS blacklist of passcodes was obtained via brute-force. To test if a PIN is blacklisted, one only needs to try it and see if a warning appears. During the initial iOS setup there is no rate-limiting in place, which enabled us to quickly test all possible PINs.

We constructed a device to automate this process using a Raspberry Pi equipped with a camera. The device emulates a USB keyboard, which is connected to the iPhone. After entering a PIN, the camera takes a photo of the iPhone screen. The photo is sent to a remote server, where the presence of the blacklist warning is detected by extracting the text in the photo.

More details and photos of the PIN extraction device can be found here: Extracting iOS' Passcode Blacklist

Responsive image

Datasets



Blacklists

In our study, we tested various blacklists in enforcing and non-enforcing settings. Besides Apple's 4-digit and 6-digit blacklists, we also created data-driven blacklists that are significantly (10x) smaller (27 PINs) and (10x) larger (2740 PINs) than the iOS 4-digit blacklist. To foster future research on this topic, we publish some of the studied blacklists here.

Apple's iOS Passcode blacklist was last updated with iOS 10.3 (14E277). Our data-driven blacklists, are based on PINs released by Daniel Amitay in 2011 and are partly reordered using a Markov model.
# Name Source Length Blacklisted Download
1 iOS-4-digit Apple iOS 4-digit 274
2 iOS-6-digit Apple iOS 6-digit 2910
3 DD-4-digit-27 Top Amitay 4-digit 27
5 DD-4-digit-2740 Top Amitay 4-digit 2740

PIN Datasets

Before our user study, the most realistic set of 4-digit PINs was from 2011, where Daniel Amitay developed the iOS application "Big Brother Camera Security." The app mimicked a lock screen allowing users to set a 4-digit PIN. Amitay anonymously and surreptitiously collected 4-digit PINs (204 432). As there was no similar 6-digit PIN data available to inform our attacker, we relied on 6-digit PINs extracted from the RockYou password leak, similar to Bonneau et al. (2012) and Wang et al. (2017). PINs are extracted from consecutive sequences of exactly n-digits in leaked password data. By following this method, we extracted 6-digit PINs from the RockYou password leak, which we refer to as RockYou-6-digit (2 758 490 PINs). For comparison, we also provide a 4-digit version of the RockYou dataset (1 780 587 PINs).

Our attacker guesses PINs in decreasing probability order based on the Amitay-4-digit and RockYou-6-digit datasets. When two or more PINs share the same frequency, i.e., it is not possible to directly determine a guessing order, we ordered those PINs using a Markov model.
# Name Source Length PINs Download
1 Amitay-4-digit Amitay 4-digit 204 432
2 RockYou-4-digit RockYou 4-digit 1 780 587
3 RockYou-6-digit RockYou 6-digit 2 758 490

User-Chosen PINs

We conducted a user study of 4- and 6-digit PINs using Amazon Mechanical Turk (MTurk) with (n=1220) participants over a period of three weeks. To mimic the PIN creation process in our browser-based study, participants were restricted to mobile devices by checking the User-Agent string. We applied a 9-treatment, between-subjects study protocol for the PIN selection criteria, e.g., 4- vs. 6-digit with or without blacklisting. At the end of the study, we collected 851 and 369 PINs, 4- and 6-digits, respectively. These PINs were all selected, confirmed, and recalled. We additionally recorded all intermediate PIN selections, such as what would happen if a selected PIN was not blacklisted and the participant did not have to select a different PIN.

We share this dataset of user-chosen PINs with other research institutions upon request. Contact details () are published along with the paper.


Technical Paper


Our work will appear at the 41st IEEE Symposium on Security and Privacy. A preprint of the paper is available here:

Download the Paper


Abstract

In this paper, we provide the first comprehensive study of user-chosen 4- and 6-digit PINs (n=1220) collected on smartphones with participants being explicitly primed for the situation of device unlocking. We find that against a throttled attacker (with 10, 30, or 100 guesses, matching the smartphone unlock setting), using 6-digit PINs instead of 4-digit PINs provides little to no increase in security, and surprisingly may even decrease security.

We also study the effects of blacklists, where a set of "easy to guess" PINs is disallowed during selection. Two such blacklists are in use today by iOS, for 4-digits (274 PINs) as well as 6-digits (2910 PINs). We extracted both blacklists compared them with four other blacklists, including a small 4-digit (27 PINs), a large 4-digit (2740 PINs), and two placebo blacklists for 4- and 6-digit PINs that always excluded the first-choice PIN. We find that relatively small blacklists in use today by iOS offer little or no benefit against a throttled guessing attack. Security gains are only observed when the blacklists are much larger, which in turn comes at the cost of increased user frustration. Our analysis suggests that a blacklist at about 10% of the PIN space may provide the best balance between usability and security.


BibTeX

If you want to cite the dataset or the paper, please use the following BibTeX entry:

@inproceedings{markert-20-pin-blacklist,
    author = {Markert, Philipp and Bailey, Daniel V. and Golla, Maximilian and D\"{u}rmuth, Markus and Aviv, Adam J.},
    title = {{This PIN Can Be Easily Guessed: Analyzing the Security of Smartphone Unlock PINs}},
    booktitle = {IEEE Symposium on Security and Privacy},
    year = {2020},
    series = {SP~'20},
    pages = {},
    address = {San Francisco, California, USA},
    month = may,
    publisher = {IEEE}
}