Protecting Against SSH Brute Force Attacks via Faillock

Modern network infrastructure faces constant probing from distributed botnets targeting the Secure Shell (SSH) interface. Within high-concurrency environments such as energy grid control systems, municipal water SCADA networks, or cloud-scale data centers: a successful brute force compromise bypasses external firewall logic by exploiting valid protocol negotiation to exhaust credential possibilities. The pam_faillock module serves as the primary defensive mechanism for tracking failed authentication attempts and locking accounts independently of the application layer. This system provides a robust framework to mitigate password spraying and dictionary attacks while maintaining low latency during the authentication handshake. Unlike its predecessor, pam_tally2, the Faillock interface stores temporary metadata in the /run/faillock/ directory. This volatile memory space prevents disk I/O bottlenecks and minimizes overhead during high-volume attack surges; effectively ensuring that security logging does not introduce signal-attenuation in system responsiveness. By implementing Faillock Brute Force Defense, architects transition from reactive firewall rules to proactive, identity-aware resource protection.

Technical Specifications

The Configuration Protocol

Environment Prerequisites:

Before execution, verify the system is running a modern Linux kernel (4.18 or higher) to support the necessary filesystem attributes for volatile lock storage. The administrator must possess sudo or root privileges. Ensure that SSH is configured to use PAM by verifying the directive UsePAM yes is active in the /etc/ssh/sshd_config file. In mission-critical environments like water treatment logic controllers or energy substations: ensure an out-of-band management console (IPMI or Serial-over-LAN) is available to prevent accidental lockout during the transition.

Section A: Implementation Logic:

The theoretical foundation of Faillock Brute Force Defense relies on the interception of the authentication payload at the PAM stack level. When a user submits a credential, the PAM subsystem triggers a sequence of modules. Faillock acts as a gatekeeper in the “auth” and “account” phases. In the “auth” phase, it increments a failure counter for the specific UID (User ID) if the password verification fails. If the counter exceeds a pre-defined threshold within a specific window, the module returns a “temporary failure” status, effectively dropping the connection before the system attempts further computation-heavy verification. This idempotent approach ensures that repeated failed attempts do not increase the CPU overhead linearly. In high-traffic scenarios, this prevents the system from reaching a state of high thermal-inertia where the processor becomes saturated by processing illegitimate cryptographic hashes.

Step-By-Step Execution

Verify Current Authentication Profile

Execute the command authselect current to determine the active PAM profile. If the system is managed by authselect, manual edits to PAM files may be overwritten unless a custom profile is created.
System Note: This command queries the existing symbolic links in /etc/pam.d/. It ensures the administrator understands the current encapsulation of security rules before introducing new logic-controllers into the stack.

Configure Global Faillock Parameters

Open the primary configuration file located at /etc/security/faillock.conf. Uncomment and modify the following parameters: deny = 5, unlock_time = 900, and even_deny_root.
System Note: Modifying this file changes the behavior of the pam_faillock.so binary. Setting even_deny_root is critical for high-security infrastructure to prevent specialized brute force attacks against the administrative account; however; it increases the risk of total system lockout if not managed via a secondary whitelist.

Enable Faillock via Authselect

Run the command authselect select with-faillock followed by authselect apply-changes. This automatically injects the Faillock module into the correct positions within the /etc/pam.d/system-auth and /etc/pam.d/password-auth stacks.
System Note: The authselect tool maintains the integrity of the PAM stack. It ensures that the pam_faillock.so module is called before and after the pam_unix.so module: allowing the system to track failures before a user even attempts a password.

Inspect the PAM Stack Order

Manually audit /etc/pam.d/system-auth using cat or vim. Ensure that pam_faillock.so appears at the top of the “auth” section with the preauth argument: and again later with the authfail argument.
System Note: Improper ordering in the PAM stack leads to a throughput collapse where valid users are erroneously blocked or failed attempts are never registered. The preauth call checks if the user is already locked: preventing further processing if the threshold is met.

Manual Reset of Account Locks

If a legitimate user is locked out, use the command faillock –user –reset to clear the failure records. To view current failed attempts, execute faillock –user .
System Note: This interacts directly with the files in /run/faillock/. Clearing these files resets the tally metadata. Because these records reside in a temporary filesystem, a hard reboot of the physical hardware will also clear all active lockouts.

Section B: Dependency Fault-Lines:

A common failure point in Faillock deployment is the conflict between local authentication and remote directory services such as LDAP or Active Directory. If the network experiences significant packet-loss or high latency, the timeout for the directory lookup might exceed the PAM session limit. In such cases: Faillock might register a system error as a failed login attempt, leading to a cascade of account lockouts. Furthermore; if the /run partition (tmpfs) is mounted as read-only or is full: the module will fail “open” or “closed” depending on the configuration: potentially allowing unlimited attempts or blocking all logins entirely. Administrators must ensure that monitoring tools like zabbix-agent or prometheus-node-exporter track the utilization of the /run filesystem to prevent such bottlenecks.

THE TROUBLESHOOTING MATRIX

Section C: Logs & Debugging:

When a lockout occurs, the primary diagnostic trail is located in /var/log/secure (RHEL/CentOS) or /var/log/auth.log (Debian/Ubuntu). Search for strings containing “pam_faillock” or “Consecutive login failures”. Use the command journalctl -u sshd to correlate SSH daemon events with PAM disruptions.

If the faillock command returns no data but the user cannot log in: investigate the UID range. By default, Faillock may ignore UIDs below 1000 unless even_deny_root is explicitly set. Use ls -l /run/faillock/ to verify if the system is creating per-user binary blobs. If these files are missing: the module is not being triggered in the “auth” stack correctly. Check for a “success” flag in a previous PAM module that might be terminating the stack evaluation prematurely: a common issue when pam_permit.so or pam_sss.so is misconfigured with “sufficient” instead of “required” control flags.

OPTIMIZATION & HARDENING

– Performance Tuning: To minimize the overhead of locking on high-traffic jump hosts: set the dir parameter in faillock.conf to point to a high-speed NVMe mount if volatile memory is insufficient. However: for 99% of use cases, the default tmpfs location provides the lowest latency. Adjust the fail_interval to a narrower window (e.g., 600 seconds) to ensure that the cumulative throughput of authentication attempts is strictly throttled without indefinitely punishing users for occasional typos.

– Security Hardening: Implement an IP-based whitelist using pam_access.so in conjunction with Faillock. By editing /etc/security/access.conf: you can define specific subnets (e.g., the local NOC or Engineering VLAN) that are exempt from lockout policies. This ensures that even during a massive brute force payload delivery: core systems remain accessible from trusted internal nodes. Additionally: use chmod 0600 on all Faillock configuration files to prevent local attackers from identifying the lockout thresholds and timing their attacks to stay just below the limit.

– Scaling Logic: For distributed environments: leverage idempotent configuration management tools like Ansible to deploy faillock.conf across the entire fleet. While Faillock stores data locally on each node: centralization can be achieved by streaming PAM logs to a SIEM (Security Information and Event Management) platform. This allows for the correlation of lockout events across multiple geographic zones: identifying if a specific IP address is attempting a horizontal brute force attack across the entire infrastructure.

THE ADMIN DESK

How do I unlock a user without clearing their history?
You cannot selectively unlock without clearing the tally in the standard faillock utility. To restore access, use faillock –user –reset. This clears the volatile records in /run/faillock/, allowing the user to attempt authentication again immediately.

Does Faillock protect against local console brute force?
Yes. Because Faillock is integrated into the system-auth and password-auth PAM stacks: it protects all entry points including SSH, TTY, and local GUI logins. Only services that bypass PAM entirely would remain vulnerable to credential exhaustion.

Will a system reboot clear all active lockouts?
Yes. Faillock defaults to storing failure records in /run/faillock/, which is a tmpfs (RAM-based) filesystem. Physical power cycles or reboots wipe this directory: effectively resetting all lockout counters to zero across the entire local user database.

Can I whitelist specific IP addresses from Faillock?
Faillock itself does not natively filter by IP. To achieve this: use pam_access.so before the Faillock calls in the PAM stack. This allows you to define trusted bypass rules based on the source network or originating hostname.