For general information on LKRG, please visit its main wiki page.

The examples below were written/recorded in March 2017 through April 2018. We released LKRG 0.0 in January 2018. The examples are still mostly valid until LKRG 0.7 released in July 2019. We've since reworked the sysctl's for LKRG 0.8+ released in June 2020, and some LKRG messages are now different.

After successfull compilation of the project, “output” directory is created with the following files:

• p_lkrg.ko – Linux Kernel Runtime Guard loadable module

This is the core of the project. Module itself accepts one parameter which determines the default logging level. By default, it is “active” level (levels will be described in details in the next section). Each level of logging is represented by numbers. By providing numbers between 0-4 or 0-6 (if compiled with debugging option) LKRG will immediately use this level even during installation in the kernel memory. It is a very useful option for debugging purpose when LKRG encounters a problem during installation in the system:

Installation of LKRG is exactly the same as loading normal kernel module. As soon as system is installed it starts the work. If default logging level is used, LKRG produces one short sentence saying that system is clean unless corruptions are detected. Otherwise LKRG informs where corruption happened, e.g.:

If system is started with more detailed log level, more information will be visible, e.g.:

(this is not the latest version of the project so might not have acquired all information)

If debugging compilation is available, it may produce enormous amount of information regarding the running system (including every function name where LKRG is entering and leaving, etc.). Sometimes even default “active” log level might be too noisy (every time when LKRG checks the system, one-line status information will be produced in case the system is clean). That’s why “none” log level option was introduced. System at that log level is quiet until abnormal situation is detected by LKRG. In that case only alerting logs will be produced.

The project has built in a sysctl interface which enables the interaction between the administrator and LKRG. By default 4 different options are available:

root@pi3-ubuntu:~/p_lkrg-main# sysctl -a|grep lkrg
lkrg.block_modules = 0
lkrg.clean_message = 1
lkrg.force_run = 0
lkrg.log_level = 1
lkrg.random_events = 1
lkrg.timestamp = 15

1. Blocking module functionality (lkrg.block_modules) – only two options are available:
   • 0 – do NOT lock the kernel and allow to load kernel module
   • 1 – lock the kernel and do NOT allow to load kernel module
2. Printing “System is clean!” message (lkrg.clean_message) – only two options are available:
   • 0 – do NOT print “System is clean!” message regardless of log_level value
   • 1 – if log_level value allows it, print “System is clean!” message
3. Force (lkrg.force_run) – forces LKRG to run integrity function right now. It is always visible as 0 number. Nevertheless, if you set it to 1, the integrity checking function will be immediately fired and value restored to 0 again
4. log level (lkrg.log_level) – it might be a number between 0-4 or 0-6 (if debugging compilation was used). A strong debug provides very useful data to identify where could be a specific problem with LKRG (if it ever appears). Unfortunately, it produces tons of logs per execution and must be used only for debugging purpose, not as a normal run.
5. Random events (lkrg.random_events) - only two options are available:
   • 0 – do NOT perform integrity checking on the random events (perform it only at regular intervals configured by lkrg.timestamp)
   • 1 – perform integrity checking on the random events (as well as at the regular intervals)
6. timestamp (lkrg.timestamp) – changes how often kernel timer will be launched (kernel timer periodically calls integrity function). It can’t be less than 5 seconds (not to eat too much system resources) and not more than 1800 seconds (half an hour) – not to be silent for too long

Here is an example of how to use the blocking module functionality:

root@pi3-ubuntu:~/p_lkrg-main# sysctl lkrg.block_modules=1
lkrg.block_modules = 1
root@pi3-ubuntu:~/p_lkrg-main# rmmod btrfs
rmmod: ERROR: Module btrfs is not currently loaded
root@pi3-ubuntu:~/p_lkrg-main# modprobe btrfs
modprobe: ERROR: could not insert 'btrfs': Operation not permitted
root@pi3-ubuntu:~/p_lkrg-main# modprobe btrfs
modprobe: ERROR: could not insert 'btrfs': Operation not permitted
root@pi3-ubuntu:~/p_lkrg-main# modprobe btrfs
modprobe: ERROR: could not insert 'btrfs': Operation not permitted
root@pi3-ubuntu:~/p_lkrg-main# sysctl lkrg.block_modules=0
lkrg.block_modules = 0
root@pi3-ubuntu:~/p_lkrg-main# modprobe btrfs
root@pi3-ubuntu:~/p_lkrg-main# rmmod p_lkrg

It is possible to compile LKRG project with “hiding” feature. In that case LKRG exports an extra option via sysctl interface to hide itself in the system:

1. Hiding functionality (lkrg.hide) – if “hiding” compilation has been done, LKRG has an extra functionality of (un)hiding itself:
   • 1 – hide LKRG (if it is not already hidden)
   • 0 – unhide LKRG (if it is not already unhidden)

Unhiding functionality is a bit of complex operation if module was previously hidden. LKRG module object is synthetically created (which implies some referenced values are incorrect) and it is linked again in proper linked lists. After all, you can safely unload LKRG but you must use ”–force” option. Example:

root@pi3-ubuntu:~/p_lkrg-main# sysctl -a |grep lkrg
lkrg.block_modules = 0
lkrg.clean_message = 1
lkrg.force_run = 0
lkrg.hide = 0
lkrg.log_level = 0
lkrg.random_events = 1
lkrg.timestamp = 15
root@pi3-ubuntu:~/p_lkrg-main# lsmod|grep p_
p_lkrg                 94208  0
root@pi3-ubuntu:~/p_lkrg-main# sysctl lkrg.hide=1
lkrg.hide = 0
root@pi3-ubuntu:~/p_lkrg-main# lsmod|grep p_
root@pi3-ubuntu:~/p_lkrg-main# sysctl lkrg.hide=0
lkrg.hide = 1
root@pi3-ubuntu:~/p_lkrg-main# lsmod|grep p_
p_lkrg                 94208  -2
root@pi3-ubuntu:~/p_lkrg-main# rmmod --force p_lkrg
root@pi3-ubuntu:~/p_lkrg-main#

Linux boots UP kernel as long as the system has only 1 (V)CPU with 1 core. However, if hot-plugging (V)CPU is supported, OS during run-time might decide to boot SMP kernel without restarting the entire OS when a new (V)CPU or core appears. This is very critical situation from the LKRG point of view – mainly because hashes from the most sections will change. Some of the functionalities are different for UP and for SMP. LKGP takes this into account and correctly rebuilds internal database as soon as this situation happens. It can be seen as follows:

Let’s add 1 more VCPU:

New CPU was correctly added but it is still in offline state:

We must change cpu1 status to online to make it work and initialize. If we do it, we can see following:

As soon as 2nd CPU became online, kernel switched to SMP code, and boot itself on SMP configuration via “smpboot” process. This operation also forced some modules to be reloaded and since LKRG was configured with enabled “blocking modules” feature, it blocked these operations. Additionally, LKRG added new CPU’s critical metadata to the database for individual protection. It will send now IPI to each CPU individually, forcing them to run checking routine. If we enable debugging option (doesn’t need to be strong debug), we can see in logs it is really happening (LKRG logs):

LKRG also correctly handles hot-unplugging CPUs or making them offline:

LKRG logs:

Again, if we enable the debug logs we can see offline CPU is present but not active and metadata only from 1 CPU is protected (but offline CPU’s state is still protected):