Using BTF to Build Out-of-Tree Kernel Modules with Private Struct Definitions

Introduction

OoT kernel modules often face challenges when they need access to private header struct definitions that are not available in public headers. Traditional methods to access these private headers can lead to complications and maintenance challenges. This blog post presents a PoC that demonstrates a method to write OoT kernel modules using BTF to leverage private header struct definitions. This approach aims to simplify the build process and improve maintainability.

What is BTF and Why is it in the Linux Kernel?

BPF is a technology used for network packet filtering, tracing, and monitoring within the Linux kernel. It allows users to run sandboxed programs in the kernel space, enabling powerful debugging and performance analysis capabilities. Producing BPF machine code is straightforward with a compiler that targets BPF, but writing a BPF program is more complicated due to the need to access kernel data during execution. For example, if you want to check if the IP of a given packet is your target, you need to access the structure representing the packet in your BPF program. You must navigate to the correct field by moving from the structure's address by a specific offset and interpret it correctly. This is where BTF comes into play. BTF, or BPF Type Format, is a slim and compact way to represent the structures used in the kernel, accounting for structure randomization. It provides rich type information for BPF programs, essential for accessing and manipulating kernel data structures accurately. BTF enhances the BPF ecosystem by enabling programs to understand and work with kernel data without needing explicit header files. To support BPF program development, an ecosystem has emerged, with libbpf being the key library that facilitates this. BPF programs need to be loaded into the kernel, and there is a BPF syscall for this operation. libbpf allows creating a loader program in native assembly that not only loads the program into the kernel but also links it (similar to the compiler's link process) to adapt it to the specific kernel, using BTF. Historically, Clang was the first C compiler to support the BPF target. GCC also supports the BPF target, but Clang remains the more commonly used compiler for this task. BTF focuses solely on describing data structures, which is why it is much more compact than other debugging formats like DWARF. The BTF section included in a production kernel is around 10-20MB, while DWARF info would be around 250-500MB.

Using BTF to Ease OoT Module Build and Maintenance

The PoC demonstrates how to build an OoT kernel module that requires private struct definitions by utilizing BTF. Here’s a step-by-step overview of the process:

• Search for Structure to Define: Identify the private structures and unions needed for the OoT module from the Linux headers.
• Collect All Structures and Unions: Gather all relevant structures and unions from the Linux headers.
• Extract vmlinux from Bootable Image: Extract the vmlinux file from a bootable kernel image, which contains the BTF information.
• Extract Structures from BTF: Use BTF to extract the required structures from the vmlinux file.
• Filter BTF Extracted Structures: Filter out the structures that are already declared in public headers to avoid duplication.
• Produce Header File: Generate a header file containing the necessary structures and unions.
• Build Kernel Module: Use a customized Makefile and scripts to build the kernel module with the generated header file.

The PoC includes:

• A customized Makefile that runs scripts to prepare the environment.
• Module source code that marks structures with //BTF_INCLUDE to indicate they need to be imported.
• Scripts to ensure consistency with existing structure declarations in public headers.
• Scripts to handle dependencies and recursively extract related structures without redeclaring existing ones.

Conclusion

This PoC showcases a functional solution for using BTF to build OoT kernel modules with private struct definitions. It demonstrates how BTF can be used to retrieve structure information about non-public definitions. While this PoC is not intended to promote the use of non-public structures in OoT modules, it acknowledges that sometimes this is unavoidable. Using BTF for this purpose can significantly increase the maintainability of the OoT kernel module across different kernel versions.