User:Pietinger/Tutorials/Kernel Hardening with KSPP
Even though this page is in the user namespace, corrections and additions are much appreciated! This is simply wiki policy, this page can be moved to the main wiki as soon as it achieves critical mass more.
No, KSPP does not mean Kerbal Space Program Project !
Tutorial: Kernel Hardening with KSPP
This tutorial shows an easy way to harden your kernel. Only our default sys-kernel/gentoo-sources will be used.
If you think you dont need it because you have Hardened-Sources you are wrong. Hardened-Sources has no hardened kernel; you must harden your kernel by yourself !
Maybe this article will become obsolete when kernel version 6.7 is available, because then we will have this:
https://lore.kernel.org/lkml/202310300946.C0E11C5@keescook/
Also described here: https://www.phoronix.com/news/Linux-6.7-Hardening
Update: I said that it could become obsolete with kernel version 6.7. Since I have now installed 6.7.0, I immediately looked at the file /usr/src/linux-6.7.0-gentoo/kernel/configs/hardening.config and the file /usr/src/linux-6.7.0-gentoo/arch/x86/configs/hardening.config and realized that only the most primitive options are activated.
This means that the KSPP page is much more comprehensive and should still be our target.
Link to KSPP
In every case you need this: https://kspp.github.io/Recommended_Settings
(Old link: https://kernsec.org/wiki/index.php/Kernel_Self_Protection_Project/Recommended_Settings )
Print the whole page or copy it into a text-file; you will need it later.
The following patch has not (yet) been included in the kernel, but contains more detailed explanations of some options (for reading only): https://lore.kernel.org/kernel-hardening/1536516257-30871-1-git-send-email-s.mesoraca16@gmail.com/
In this video from the "Linux Security Summit 2021" Kees Cook (who also updates this KSPP Page) explains why it is so important to harden your kernel and also explains the progress made from Linux version 5.3 until 5.15: https://www.youtube.com/watch?v=-Binxid8t_8
(Maybe interesting: Kees Cook & Gustavo A.R. Silva explain the "Progress On Bounds Checking in C and the Linux Kernel" (Linux Security Summit North America 2023): https://www.youtube.com/watch?v=V2kzptQG5_A )
Transfer KSPP settings into your kernel
Of course you can set all these settings manually into your kernel. This you have to do if you dont use Gentoo-Sources. With Gentoo-Sources there is an easy way ... Maybe you have already seen this kernel setting:
Gentoo Linux --->
[ ] Kernel Self Protection Project
After enabling this option maybe you will see this (the second option only appears when you have enabled the first):
--- Kernel Self Protection Project
[*] Enable Kernel Self Protection Project Recommendations
[ ] X86_64 KSPP Settings
Maybe you dont have these two options. Why ?
If you read all recommended settings in KSPP Homepage you will see there are many kernel options which must be disabled. Disabling a kernel option with a Kconfig file is not possible. The only way to proof if something is disbled is a check in Kconfig with "... !options". ! means: NOT. Now look into /usr/src/linux/distro/Kconfig. You will see in the second half of this file a line like this:
/usr/src/linux/distro/Kconfig of Kernel Version 6.6.1depends on GENTOO_LINUX && !ACPI_CUSTOM_METHOD && !COMPAT_BRK && !PROC_KCORE && !COMPAT_VDSO && !KEXEC && !HIBERNATION && !LEGACY_PTYS && !X86_X32_ABI && !MODIFY_LDT_SYSCALL && GCC_PLUGINS && !IOMMU_DEFAULT_DMA_LAZY && !IOMMU_DEFAULT_PASSTHROUGH && IOMMU_DEFAULT_DMA_STRICT && SECURITY && !ARCH_EPHEMERAL_INODES && RANDSTRUCT_PERFORMANCE
If one of these options are enabled this line cause a hiding of this option. For example if you have enabled Hibernation you will not see the kernel option "Enable Kernel Self Protection Project Recommendations".
But I need Hibernation
If you now think that you can't use this because you absolutely need “hibernation”, then there is a little trick that you can use to get everything anyway:
1. Disable hibernation (and do everything mentionend in the next chapter)
2. Enable these two settings (and everything else you want to enable or disable)
3. Exit “make menuconfig” with save
4. Go back to “make menuconfig” and disable the two settings -> Yes, all settings through these two REMAIN.
5. Enable hibernation again
(If you have any questions, please read the short chapter: User:Pietinger/Tutorials/Manual_kernel_configuration#Do_I_need_a_.22make_defconfig.22_before_I_start_.3F )
Course of action
1. First of all disable all kernel options with the description in the link you have now. Yes, also "DEVMEM". If you search for "DEVKMEM" you will not find it if you have kernel 5.15 (or newer); kernel developer have revoked it. Do not disable IA32 if you have a Multilib-System ! Do not disable Module-Support if you have kernel modules enabled with {M}. You can disable Module-Support only if you have a monolithic kernel.
2. If you have problems to find all necessary options do all settings described in the next chapter.
3. Now you are able to enable MANY options with these two options in our Gentoo section. Cancel all these many options (best with content of usr/src/linux/distro/Kconfig; but you can use also <HELP>) on your print output (or in your text file).
4. Check if there are some options you dont have enabled automatically. A diff between KSPP version 2023-10-20 and our Gentoo settings for kernel version 6.6.1 will show these missing setings - further updates to the KSPP version from 2024-04-26 can be found in the chapter below:
1) CONFIG_DEBUG_VIRTUAL=y 2) # Enable "lockdown" LSM for bright line between the root user and kernel memory. CONFIG_SECURITY_LOCKDOWN_LSM=y CONFIG_SECURITY_LOCKDOWN_LSM_EARLY=y CONFIG_LOCK_DOWN_KERNEL_FORCE_CONFIDENTIALITY=y 3) (automatically enabled in .../distro/Kconfig of kernel version 6.9) # Enable trapping bounds checking of array indexes (since v5.11). All the other UBSAN checks should be disabled. CONFIG_UBSAN=y CONFIG_UBSAN_TRAP=y CONFIG_UBSAN_BOUNDS=y CONFIG_UBSAN_SANITIZE_ALL=y # CONFIG_UBSAN_SHIFT is not set # CONFIG_UBSAN_DIV_ZERO is not set # CONFIG_UBSAN_UNREACHABLE is not set # CONFIG_UBSAN_BOOL is not set # CONFIG_UBSAN_ENUM is not set # CONFIG_UBSAN_ALIGNMENT is not set # This is only available on Clang builds, and is likely already enabled if CONFIG_UBSAN_BOUNDS=y is set: CONFIG_UBSAN_LOCAL_BOUNDS=y 4) # Do not ignore compile-time warnings (since v5.15) CONFIG_WERROR=y 5) # Disable DMA between EFI hand-off and the kernel's IOMMU setup. CONFIG_EFI_DISABLE_PCI_DMA=y 6) # Enable feeding RNG entropy from TPM, if available. CONFIG_HW_RANDOM_TPM=y 7) # Wipe RAM at reboot via EFI. CONFIG_RESET_ATTACK_MITIGATION=y 8) # This needs userspace support, and will break "regular" distros. See: https://github.com/tych0/huldufolk CONFIG_STATIC_USERMODEHELPER=y 9) # Block TTY stuffing attacks (this will break screen readers, see "dev.tty.legacy_tiocsti" sysctl below). # CONFIG_LEGACY_TIOCSTI is not set 10) # Keep root from altering kernel memory via loadable modules. # CONFIG_MODULES is not set # But if CONFIG_MODULE=y is needed, at least they must be signed with a per-build key. CONFIG_DEBUG_SET_MODULE_RONX=y (prior to v4.11) CONFIG_STRICT_MODULE_RWX=y (since v4.11) CONFIG_MODULE_SIG=y CONFIG_MODULE_SIG_FORCE=y CONFIG_MODULE_SIG_ALL=y CONFIG_MODULE_SIG_SHA512=y CONFIG_MODULE_SIG_HASH="sha512" CONFIG_MODULE_SIG_KEY="certs/signing_key.pem" x86_64 11) # Remove additional (32-bit) attack surface, unless you really need them. # CONFIG_COMPAT is not set # CONFIG_IA32_EMULATION is not set # CONFIG_X86_X32 is not set # CONFIG_X86_X32_ABI is not set 12) # Disallow allocating the first 64k of memory. CONFIG_DEFAULT_MMAP_MIN_ADDR=65536 13) # Straight-Line-Speculation CONFIG_SLS=y (MY NOTE: this option was renamed with 6.9. to: CONFIG_MITIGATION_SLS)
My comments:
1. This costs some performance. Decide yourself.
2. See the big warning !
3. Enable it.
4. This could make some problems when doing cross compile (I dont do and therefore I have enabled it).
5. See the big warning !
6. Only senseful if you have such a hardware module (I dont have; so its disabled).
7. See the big warning !
8. Enable it and make sure that the PATH is empty. But test it if everything still works afterwards, because I got a report that iptables does not work properly anymore if it has to load netfilter modules (not verified because I have all netfilter modules statically configured in the kernel and therefore have no problem with iptables AND this option).
9. Read the <Help> and test it ... ;-)
10. I have disabled the module support and have a monolithic kernel ... after I found out which modules I need for my firewall ;-) If you need module support (e.g. for nvidia) sign your kernel modules.
11. You can do this only if you have a No-Multilib system. See also: User:Pietinger/Tutorials/Manual_Configuring_Kernel_Version_6.6#32-bit_support
12. This is very senseful and we have done it already.
13. If security is more important than performance enable it (I have it enabled) - All mitigitations cost some performance; dont use it for a gaming system:
[*] Mitigations for speculative execution vulnerabilities --->
[*] Mitigate Straight-Line-Speculation
Since 2023-10-20 we have new recommendations in KSPP. One of them is enabling LOCKDOWN. Do this only if you have signed your Kernel modules (see here: Signed_kernel_module_support ), OR you have a monolithic kernel (see more here: Kernel_Modules#Going_completely_.22module-less.22 ) !
If you boot a stub-kernel directly from UEFI, these settings could harm your boot sequence (BIOS hangs): CONFIG_EFI_DISABLE_PCI_DMA and CONFIG_RESET_ATTACK_MITIGATION. Best is to try only one of them and (if ok) then enable the second option.
5. Dont forget some settings in SYSCTL. You can copy it from KSPP directly in (just add them at the end):
root #nano -w /etc/sysctl.confIf you have problems when emerging some packages (e.g. dev-lang/go) because of this:
/etc/sysctl.confkernel.yama.ptrace_scope = 3
Then set it to 2 (or even 1). Please see more in: /usr/src/linux/Documentation/admin-guide/LSM/Yama.rst
6. In KSPP there are also some settings via kernel command line parameter, like e.g.: "nosmt" which would disable SMT (Intel call it "Hyperthreading"). In my point of view this is only necessary if it is a server and you have virtual machines. A desktop doesnt need it. But it is your decision ! Another kernel command line parameter: "slab_nomerge # Disable slab merging (makes many heap overflow attacks more difficult)" can be set also directly in the kernel configuration; see in chapter KSPP 2024-04-26.
Minimum kernel configuration to get both KSPP options
... for a X86_64 system. You must enable "expert users" to be able to reach all necessary options. After enabling "expert users" check if "Framebuffer Console support" still is set. If not enable it again. Now you have to DISABLE and ENABLE all these:
Kernel 5.15
General setup --->
[*] Configure standard kernel features (expert users) --->
Processor type and features --->
[ ] Enable vsyscall emulation
[ ] /dev/cpu/*/msr - Model-specific register support
[ ] kexec system call
[ ] Enable the LDT (local descriptor table)
Power management and ACPI options --->
[ ] Hibernation (aka 'suspend to disk')
General architecture-dependent options --->
[*] GCC plugins --->
Memory Management options --->
[ ] Disable heap randomization
Device Drivers --->
Character devices --->
[ ] Legacy (BSD) PTY support
Graphics support --->
Console display driver support --->
[*] Framebuffer Console support
[*] IOMMU Hardware Support --->
IOMMU default domain type (Translated - Strict) --->
File systems --->
Pseudo filesystems --->
[ ] /proc/kcore support
Security options --->
[*] Enable different security models
Kernel 6.1
Do all settings from 5.15 and ADDITIONALLY:
Security options --->
Kernel hardening options --->
Randomize layout of sensitive kernel structures (Limit randomization of structure layout to cache-lines) --->
Kernel 6.6
Do all settings from 5.15 and 6.1. With Kernel 6.6 the option "KEXEC" was moved from "Processor type and features" to "General setup":
General setup --->
Kexec and crash features --->
[ ] Enable kexec system call
[ ] Enable kexec file based system call
[ ] kernel crash dumps
Bad option
If you have enabled the really "bad" option "DEBUG_FS" it could be possible you must disable another option, because "ACPI_CUSTOM_METHOD" depends on "DEBUG_FS"; but it is much more better to disable "DEBUG_FS" (then you dont have "ACPI_CUSTOM_METHOD"). If you dont have enabled "DEBUG_FS" you will find this option only by pressing z and it is disabled by default (and you have nothing to do here):
Power management and ACPI options --->
[*] ACPI (Advanced Configuration and Power Interface) Support --->
- - Allow ACPI methods to be inserted/replaced at run time
ACPI_CUSTOM_METHOD was completely removed with 6.9 and can therefore no longer cause problems. :-)
Slow System start ?
If you boot the first time with these new settings you will notice that your kernel needs longer to start. Before it took 1.2 seconds (until "Run /sbin/init as init process") and now kernel needs 2.5 seconds. If you think this is crazy; these settings have halved my CPU performance ... then don't worry ... and look carefully into your dmesg output and examine the timestamps. You will find something like this:
root #dmesg[...] [ 0.030070] mem auto-init: stack:all(zero), heap alloc:on, heap free:on [ 0.030071] mem auto-init: clearing system memory may take some time... [ 0.030076] software IO TLB: area num 8. [ 1.358628] Memory: 16087256K/16653428K available (14343K kernel code, 1960K rwdata, 2456K rodata, 1564K init, 2888K bss, 565912K reserved, 0K cma-reserved) [...]
(As you can see I have 16 GB RAM; machine is an Intel i7; but it doesn't matter how many cores your CPU has).
Yes, for security reasons your kernel will clear now your whole memory (with ONE Core of your CPU). If you have a 64 GB RAM machine you can expect it took aprox. 5 seconds.
KSPP 2024-04-26
All of the following options - except the first one, because disabling you have to do yourself - are automatically enabled in .../distro/Kconfig of kernel version 6.9. If you have stable 6.6 you really should enable these also.
Memory Management options --->
SLAB allocator options --->
[ ] Allow slab caches to be merged
Kernel hacking --->
Memory Debugging --->
[*] Check for invalid mappings in user page tables
[*] Enforce the page table checking by default
# If a query with "cpuid" gives you: "CET_SS: CET shadow stack = true" then enable this:
Processor type and features --->
[*] X86 userspace shadow stack
Memory Management options --->
SLAB allocator options --->
[*] Randomize slab caches for normal kmalloc
This option I have already described in User:Pietinger/Tutorials/Manual_Configuring_Kernel_Version_6.6:
# If a query with "cpuid" gives you: "IBRS/IBPB: indirect branch restrictions = true" then enable this:
Processor type and features --->
[*] Indirect Branch Tracking
Two options are only impportant if you use SysReq (I have it disabled) and module support (I am using a monolithic kernel):
CONFIG_MAGIC_SYSREQ_DEFAULT_ENABLE=176 CONFIG_MODULE_FORCE_LOAD is not set
KSPP 2024-06-17
This update includes:
- Renaming all mitigations to the new names used in kernel version 6.9: CONFIG_MITIGATION_* (see also: User:Pietinger/Experimental/Manual_Configuring_Kernel_Version_6.9 )
- The recommendation to use CONFIG_RANDSTRUCT_FULL=y ... we cannot use because Gentoo (CONFIG_GENTOO_KERNEL_SELF_PROTECTION_COMMON) wants CONFIG_RANDSTRUCT_PERFORMANCE=y
- Some typos have been corrected in some commands in the chapter "sysctls"
More Hardening Options
I deactivate the Magic SysRq key completely:
Kernel hacking --->
Generic Kernel Debugging Instruments --->
[ ] Magic SysRq key
When Google disables IO_URING then I do the same ;-) see more here: [[1]]
General setup --->
[*] Configure standard kernel features (expert users) --->
[ ] Enable IO uring support
With kernel version 6.9 we have a new option for UBSAN: CONFIG_UBSAN_SIGNED_WRAP. It's there for development testing, but I have enabled it nevertheless. Together with the old options it looks like this:
Kernel hacking --->
Generic Kernel Debugging Instruments --->
[*] Undefined behaviour sanity checker --->
--- Undefined behaviour sanity checker
[*] Abort on Sanitizer warnings (smaller kernel but less verbose)
[*] Perform array index bounds checking
[ ] Perform checking for bit-shift overflows
[ ] Perform checking for integer divide-by-zero
[*] Perform checking for signed arithmetic wrap-around
[ ] Perform checking for non-boolean values used as boolean
[ ] Perform checking for out of bounds enum values
With kernel version 6.11 we have a new option that will certainly be included in KSPP soon.
Memory Management options --->
SLAB allocator options --->
[*] Support allocation from separate kmalloc buckets
With kernel version 6.11.3 we have a new option that will certainly be included in KSPP soon. An explanation can be found here: [[2]]
Security options --->
Allow /proc/pid/mem access override (Never) --->
Check it with kernel-hardening-checker
At the moment it is not stable, so you must add this package to your /etc/portage/package.accept_keywords
root #emerge -pv kernel-hardening-checkerStart it with:
root #kernel-hardening-checker -c /usr/src/linux/.configDo not worry if some points are displayed as "Fail". You can only have AMD or Intel MCE ... or IOMMU ... ;-) See my result:
[+] Kconfig file to check: /usr/src/linux/.config
[+] Detected architecture: X86_64
[+] Detected kernel version: 6.1
[+] Detected compiler: GCC 130201
option name | type |desired val | decision | reason | check result
CONFIG_BUG |kconfig| y |defconfig | self_protection | OK
CONFIG_SLUB_DEBUG |kconfig| y |defconfig | self_protection | OK
CONFIG_THREAD_INFO_IN_TASK |kconfig| y |defconfig | self_protection | OK
CONFIG_GCC_PLUGINS |kconfig| y |defconfig | self_protection | OK
CONFIG_IOMMU_SUPPORT |kconfig| y |defconfig | self_protection | OK
CONFIG_STACKPROTECTOR |kconfig| y |defconfig | self_protection | OK
CONFIG_STACKPROTECTOR_STRONG |kconfig| y |defconfig | self_protection | OK
CONFIG_STRICT_KERNEL_RWX |kconfig| y |defconfig | self_protection | OK
CONFIG_STRICT_MODULE_RWX |kconfig| y |defconfig | self_protection | OK: CONFIG_MODULES is "is not set"
CONFIG_REFCOUNT_FULL |kconfig| y |defconfig | self_protection | OK: version >= 5.5
CONFIG_RANDOMIZE_BASE |kconfig| y |defconfig | self_protection | OK
CONFIG_VMAP_STACK |kconfig| y |defconfig | self_protection | OK
CONFIG_DEBUG_WX |kconfig| y |defconfig | self_protection | OK
CONFIG_WERROR |kconfig| y |defconfig | self_protection | OK
CONFIG_X86_MCE |kconfig| y |defconfig | self_protection | OK
CONFIG_X86_MCE_INTEL |kconfig| y |defconfig | self_protection | OK
CONFIG_X86_MCE_AMD |kconfig| y |defconfig | self_protection | FAIL: is not found
CONFIG_MICROCODE |kconfig| y |defconfig | self_protection | OK
CONFIG_RETPOLINE |kconfig| y |defconfig | self_protection | OK
CONFIG_SYN_COOKIES |kconfig| y |defconfig | self_protection | OK
CONFIG_X86_SMAP |kconfig| y |defconfig | self_protection | OK: version >= 5.19
CONFIG_X86_UMIP |kconfig| y |defconfig | self_protection | OK
CONFIG_PAGE_TABLE_ISOLATION |kconfig| y |defconfig | self_protection | OK
CONFIG_RANDOMIZE_MEMORY |kconfig| y |defconfig | self_protection | OK
CONFIG_INTEL_IOMMU |kconfig| y |defconfig | self_protection | OK
CONFIG_AMD_IOMMU |kconfig| y |defconfig | self_protection | FAIL: "is not set"
CONFIG_BUG_ON_DATA_CORRUPTION |kconfig| y | kspp | self_protection | OK
[...]
Please do not worry if some options are displayed as "fail"; we cannot configure everything as is the case with other systems (e.g. we must have KCMP activated). Others are obsolete, such as CONFIG_CHECKPOINT_RESTORE, as we now have the "CAP_CHECKPOINT_RESTORE" capability since Linux 5.9 (see also "man capabilities(7)"). You can also set/disable other options as a test (e.g. CONFIG_DEVPORT) if you do not have any applications or sensors that require this.
Interestingly, grsec and ClipOS [[3]] do the same as I do with IO_URING ... :-D
Supplement: Because some users asked what else could be configured based on the kernel-hardening-checker output, I'll add my opinion here:
What we need at Gentoo - so don't think about it:
- CONFIG_FB
- CONFIG_VT
- CONFIG_KCMP
- CONFIG_RSEQ
- CONFIG_USER_NS
- CONFIG_BLK_DEV_WRITE_MOUNTED
This could interfere with your boot process (just test it):
- CONFIG_EFI_DISABLE_PCI_DMA
- CONFIG_RESET_ATTACK_MITIGATION
Only if you use CLANG instead GCC:
- CONFIG_CFI_CLANG
- CONFIG_CFI_PERMISSIVE
Only useful if you have a hardware TPM:
- CONFIG_HW_RANDOM_TPM
Be very careful here - Check if everything runs after this:
- CONFIG_X86_IOPL_IOPERM
Good canidates:
- CONFIG_IO_URING
- CONFIG_AIO
- CONFIG_MAGIC_SYSRQ
- CONFIG_BPF_SYSCALL
- CONFIG_DEVPORT
You must decide:
- CONFIG_PROC_PAGE_MONITOR
- CONFIG_CACHESTAT_SYSCALL
This is my third try for creating a wiki article. Maybe some formatting is wrong. Because it is a tutorial and not a reference guide I ignored the standard of not writing in first or second person. Yes, my english is very poor.