So naturally I figured I should try and work out how to port WAMR to ESP32 so we can run WASM on super cheap silicon!
This article exists as a list of resources which I used along the way.
I’ve been working with ESP-IDF v4.0-rc which hopefully is not far off becoming the official 4.0 release. Version 4 uses CMake as its primary build tool, and while I’m not that keen on it I figured I’d better bite the bullet and get used to it.
There’s some advice in the documentation on
Using ESP-IDF in Custom CMake Projects
which proved quite useful … the first step was to get this building with CMake and
printing “Hello, World!”, from the
I wasted a lot of time starting from the ‘normal’ esp-idf CMake file, which wasn’t the right choice at all for this kind of project. Thanks Seon for sending a MicroPython-with-CMake example and Gus for helping get unstuck.
This also involved updating to a newer
xtensa-esp32-elf-gcc, via the weird
installer script. If you don’t want to run the install script, you can work out the
xtensa esp32 toolchain URLs here
worked for me.
WAMR is also build with CMake, which means to build the two together you have to
CMakeLists.txt with the WAMR
CMakeLists.txt. I mostly worked
from the linux and zephyr WAMR platforms and tried to work out what was needed as I
went along …
As always, I spent a maddening amount of time running in circles before finding something which worked well enough: well, it compiles and I can always clean it up later.
One very good hint I got along the way (thanks Jimmo)
is to quarantine CMake off into its own
build directory by doing something like
mkdir build cd build cmake .. make
which made a huge difference as now I could repeatably blow away the builds and start over when CMake got confused. It’s a little thing, but it makes a big difference.
The main entry point
product-mini/platforms/esp-idf/main.c is responsible for starting
up the WASM subsystem and passing it some code to run. For now, this code is just compiled
in from a byte array in
test_wasm.h, but eventually I’ll add file system support so it can
live separately. It’s closely based on the other platforms with a little bit of input
from MicroPython’s ESP32 port.
There’s a whole bunch of platform specific implementation in
(Yeah, I’m not a big fan of the directory naming and layout either.)
First I just stubbed these in with a bunch of
return 0 type code and then I’ll come back
through to add implementations for time function and threading and so on.
For very basic programs, the VM is quite tolerant of missing features, so if you’re porting to some other platform don’t get too stressed about gettime semaphores right before you say hello …
I (259) cpu_start: Compile time: Feb 5 2020 16:00:36 I (271) cpu_start: ESP-IDF: v4.0-rc I (344) cpu_start: Starting scheduler on PRO CPU. wasm_runtime_load wasm_runtime_instantiate wasm_application_execute_main Hello world! CORRUPT HEAP: multi_heap.c:431 detected at 0x3ffd4610 abort() was called at PC 0x4008969d on core 0
There are also apis to native functions in
More on those later.
There’s an example
test_wasm.h which prints
Hello world!, and getting that to execute was
a very exciting moment. But really understanding how this platform works requires a lot more!
Webassembly can be generated in a few different ways:
… and very many others
For the moment all I want to do is produce something slightly nicer than Hello World … that’s right, I want to blink an LED! So the next step is to work out how to compile my own code and pass it to WAMR for execution.
This is where WASM gets a little … confusing.
It’s surprisingly low-level, with things like
But the code in
out to be revealing, and what eventually worked for me was:
clang-8 --target=wasm32 -O3 -z stack-size=4096 -Wl,--initial-memory=65536 --sysroot=../../../../wamr-sdk/app/libc-builtin-sysroot -Wl,--allow-undefined -Wl,--export=main, -Wl,--no-threads,--strip-all,--no-entry -nostdlib -o test.wasm test.c xxd -i test.wasm > test.h
xxd is a very useful file transforming utility, which in this case is used to spit out a header-file formatted version of the wasm bytecode.
clang I just installed from the ubuntu package, but I found that
wasm2wat utility was too out of date so I installed that from
There’s a couple of open problems at this point:
printf("string!")works fine, but
printf("hello %d", n)crashes the port. The same WASM works okay in the linux port, so I’m not sure why.
Much of the point of this little effort was to get JIT working, and it turns out that that might be harder than I thought. The JIT code uses LLVM, and mainline LLVM doesn’t yet support Xtensa. There’s a LLVM for Xtensa under development so I’m building that, which on my laptop is not a fast process.
To actually run this code it’s got to target Xtensa but also the libraries themselves have to be compiled for Xtensa. Something like this might work:
cmake -DLLVM_EXPERIMENTAL_TARGETS_TO_BUILD="Xtensa" -DLLVM_TARGET_ARCH="Xtensa" -DLLVM_BUILD_LLVM_DYLIB="ON" ~/Work/llvm-project-xtensa/llvm/
After a bit more research and advice, it looks like the JIT option won’t really be practical as it’ll have to build a lot of LLVM in to the runtime … and LLVM is not small. However AoT is still on the agenda!
This article is a work in progress, and now you’re up to where I’m up to!
Check back later, or follow me on twitter for updates.
There’s also an open XTensa / FreeRTOS support issue on WAMR.