In the previous post, we introduced ESP Privilege Separation, a framework to achieve “user-kernel” separation and isolation on ESP32-C3 SoC. There are multiple ways of applying this framework to your project. This post presents a case study of integrating ESP-RainMaker, a real-world IoT application, with the ESP Privilege Separation framework.
ESP-RainMaker offers a complete ecosystem to build connected AIoT products. Please refer to this link for more details.
- Directory structure for the project
A typical project is divided into two parts, the protected app and the user app. We need to create a directory structure so that the build system can build both applications. Directory structure of the project is as follows:
| — CMakeLists.txt
| — partitions.csv
| — protected_app/
| | — main/
| | — CMakeLists.txt
| | — protected_main.c
| — user_app/
| — main/
| | — CMakeLists.txt
| | — user_code.c
| — user_config.h
| — CMakeLists.txt
Please refer to the “Getting Started” documentation for a detailed explanation regarding the directory structure.
2. Placement of the ESP-RainMaker agent
Based on the system call implementation, we have the following options to place the ESP-RainMaker agent:
- Option1: ESP-RainMaker agent in the user application
- Option2: ESP-RainMaker agent in the protected application
In this post, we will choose the second approach and we will also discuss certain implications in subsequent sections.
3. Component split between protected and user apps
The above diagram shows the overview of component split between protected and user apps. All the libraries like ESP-RainMaker, TLS stack, etc. are placed in the protected app and hence it does the bulk of the heavy lifting. User app is a lightweight application consisting of business logic.
The build system generates
app_libs_and_objs.jsonfile in the build directory, which represents all the libraries, its memory footprint, and corresponding object files included in respective applications.
4. System call implementation
As we have decided to place the ESP-RainMaker agent in the protected app, we will have to implement system calls for all the public APIs provided by ESP-RainMaker. The ESP Privilege Separation’s easy extensibility features allow you to add application-specific custom system calls, which is documented here, and we shall use the same feature here.
esp_rmaker_start() is one of the APIs exposed by ESP-RainMaker. After we move ESP-RainMaker in the protected app, all the calls to
esp_rmaker_start() in the user app’s code will go through the system call interface. The following diagram shows the call trace when
esp_rmaker_start() is called from the user app:
We can enable this by adding a custom system call as shown below:
- Firstly, we need to implement a system call wrapper in the user app. The name of this wrapper should have
usr_prefix prepended to the system call name. In this wrapper, we use
EXECUTE_SYSCALLmacro to generate a synchronous exception and land into protected space.
__NR_esp_rmaker_startis the macro for a system call number that will be generated by the build system.
NOTE: The build system maps
usr_esp_rmaker_start. This mechanism enables user app to perform a system call by calling
- Now we need to implement the protected app system call handler. This handler is called after a synchronous exception is generated. Protected handler calls the actual API and returns error code to the user space.
- To bind the user and protected system call implementation, we create a custom system call table in the example directory (examples/ rmaker_switch/components/rmaker_syscall/rmaker_syscall.tbl) . We need to define four attributes in the system call table:
1. A unique system call number.
2. A common/custom attribute indicator for a system call. Please refer to the documentation for more details.
3. System call name.
4. Protected system call handler name.
1289 common esp_rmaker_start sys_esp_rmaker_start
This system call table file is processed by the build system to create the
__NR_esp_rmaker_start macro that thus binds the user-app’s
EXECUTE_SYSCALL call to the protected app’s
The above example is just for demonstration purpose. Please refer to rmaker_syscall component for actual implementation.
5. User app implementation
After implementing these system calls, we can use almost all the public APIs provided by ESP-RainMaker in the user app. We have implemented an IoT switch application using the services added in the protected app. The code for user app is available on the GitHub repository.
From the above numbers, we can observe that the total binary size of the privilege separation application is the same as that of a traditional application.
At the expense of little static memory usage, this framework allows us to decouple the monolithic firmware into two separate and isolated applications.
- ESP-RainMaker agent initialization:
After the user app startup code registers heap and console, it hands over the control to user_main.
user_main function initializes the ESP-RainMaker agent, creates a RainMaker device, and starts the ESP-RainMaker agent in the protected app through the system call interface. The protected app manages Wi-Fi connection and provides all the RainMaker services.
2. ESP-RainMaker cloud connection:
Once Wi-Fi is successfully provisioned and connected, the protected app establishes a TLS connection with the RainMaker cloud. Now our ESP32-C3 device is ready to receive events from the cloud. The protected app receives all the cloud events and triggers user space callbacks based on the configuration.
- Frequently changing business logic can be easily deployed by independently upgrading the user app while reusing the same protected app (RainMaker kernel).
- The protected app establishes the TLS connection with the RainMaker cloud. In case of a user app crash, TLS connection is still intact and the protected app can send real-time debugging information to the RainMaker cloud.
- We implemented a system call interface exposed by the protected app to provide RainMaker services.
- We integrated the upstream ESP-RainMaker application with minimal changes.
- We compared binary sizes and memory usage of traditional ESP-RainMaker application with the ESP Privilege Separation integration.