Our clients use various programming languages. So how do we manage to create and control the entire front-end layer, regardless of the back-end stack they use?
Client-side libraries like React or Vue are able to interop with any back-end by using API’s. But what if we want to server-side render our front-end or be able to create a static site with our favorite front-end tools on the back-end of a client that uses a particular tech stack that has no tooling for it? One solution would be using WebAssembly.
WebAssembly’s portability
Besides WebAssembly’s main goal, which is offering better performance on the web, it also offers portability with its binary format, that is designed to be executable on a variety of operating systems and instruction set architectures, on the Web as well as off the Web.
Using WebAssembly, we can create a module containing whatever logic we want, that subsequently can be used to interop with various programming languages (see Wasm runtimes). As a proof of concept we tried to integrate the Handlebars templating language in a WebAssembly module. Templating languages are often implemented in different programming languages. Take Mustache, the predecessor of Handlebars for example. Originally written in Ruby, but now has implementations available for JavaScript, PHP, Python, Java, etc.
By integrating Handlebars in a WebAssembly module, we wouldn’t need to depend on specific implementation of the templating language that is compatible with the back-end tech stack of our clients. Instead, we could use WebAssembly’s portability to reuse the templating language logic across different tech stacks.
Requirements
Before starting with the Proof of Concept, we first defined a few high level requirements:
- Obviously, integration of the Handlebars templating language in a WebAssembly module.
- Rendering templates with the WebAssembly module that contains the templating language logic.
- Making the WebAssembly module interop with at least one other language besides JavaScript, which has native support for WebAssembly.
Project setup with Rust
For the realization of the Proof of Concept, we decided to use Rust since we’re already experienced with the language. The following Rust dependencies were used:
- Handlebars-rust: a Rust implementation of the Handlebars templating language.
- Serde JSON: a framework for serializing and deserializing Rust data structures. Used to serialize and deserialize Handlebars templates and data from and to the WebAssembly module.
- Wasmer: A WebAssembly runtime for executing WebAssembly on the server. Used to add WebAssembly support on other programming languages.
Proof of Concept: how to make it work
WebAssembly is still in its infancy, which makes some trivial coding challenges more difficult than you’re used to. Here are some of the challenges we encountered when realizing the Proof of Concept:
WebAssembly’s memory model
Rendering Handlebars templates with the WebAssembly module, requires passing templates and data to the module to receive the rendered HTML back. To understand how data is passed to and from a WebAssembly module, we first need to understand how WebAssembly’s memory model works.
WebAssembly has a rather simple memory model compared to modern programming languages that use a garbage collected heap. Each WebAssembly module has access to a single linear memory, that is represented by an array of bytes. JavaScript for example, can write and read to this memory, but currently only as a buffer of value types int32, int64, float32, and float64. WebAssembly’s function parameters and return values are currently also restricted by these types.
Interoperability
In order to render templates, Handlebars Rust requires strings for templates and JSON for the data. Since WebAssembly doesn’t natively support these types, some work was required to get it working.
Using JavaScript, we decided to do it as following:
1. Encode the template and the JSON data as UTF-8 byte arrays. We decided to use UTF-8 to represent strings because programming languages tend to differ in character encoding.
const postTemplateUtf8 = (new TextEncoder()).encode(postTemplate);
const jsonUtf8 = (new TextEncoder()).encode(json);2. Allocate memory given the UTF-8 byte arrays length of the template and JSON data, with the ‘alloc’ function that is exported from Rust.
const postTemplatePtr = main.alloc(postTemplateUtf8Length);
const jsonPtr = main.alloc(jsonUtf8Length);The ‘alloc’ function, that returns a pointer:
pub fn alloc(size: usize) -> *mut c_void {
let mut buffer = Vec::with_capacity(size);
let pointer = buffer.as_mut_ptr();
mem::forget(buffer);
pointer as *mut c_void
}3. Write the UTF-8 byte arrays as C strings to the WebAssembly module memory, using the pointers and UTF-8 byte arrays length.
new Uint8Array(main.memory.buffer, postTemplatePtr, postTemplateUtf8Length)
.set(postTemplateUtf8);
new Uint8Array(main.memory.buffer, jsonPtr, jsonUtf8Length)
.set(jsonUtf8);4. Pass the pointers of the template and JSON data to the WebAssembly module using the exported Rust function ‘render’.
const htmlPtr = main.render(postTemplatePtr, jsonPtr);The ‘render’ function:
pub fn render(template: *mut c_char, data: *mut c_char) -> *mut c_char {
let template = c_char_to_str(template).unwrap();
let data = c_char_to_str(data).unwrap();
let json: Value;
match serde_json::from_str(&data) {
Ok(temp_json) => json = temp_json,
Err(e) => panic!("Could not parse to json: {}", e),
};
let handlebars = Handlebars::new();
let html;
match handlebars.render_template(&template, &json) {
Ok(temp_html) => html = temp_html.into_bytes(),
Err(e) => panic!("Could not render: {}", e),
}
unsafe { CString::from_vec_unchecked(html ) }.into_raw()
}Within the ‘render’ function, the pointers for the template and JSON data are used to read the memory as C strings in order to recreate the strings. The function ‘c_char_to_str’ is used for this:
fn c_char_to_str<'a>(chars: *mut c_char) -> Result<String, Utf8Error> {
let chars = unsafe { CStr::from_ptr(chars).to_bytes().to_vec() };
match str::from_utf8(&chars) {
Ok(v) => Ok(v.to_string()),
Err(e) => Err(e),
}
}Additionally since Rust Handlebars requires the data to be JSON, Serde JSON is used to convert the data to JSON. Having the templates and data to be of the required types, Handlebars Rust can finally render the HTML string. At the end of the function, a C string is created from the HTML string and a pointer to it is returned.
5. On the JavaScript side, the pointer is used to recreate the HTML string from the WebAssembly module memory:
const memory = new Uint8Array(main.memory.buffer, htmlPtr);
const htmlBytes = [];
for (const byte of memory) {
if (byte === 0) break;
htmlBytes.push(byte);
}
const html = new TextDecoder().decode(new Uint8Array(htmlBytes));
console.log(html);The memory of the WebAssembly module is read as a C string to find the bytes for the HTML string. The bytes are then decoded back to a string and finally logged to the console.
Integration with other languages
The above example shows how the WebAssembly module works with JavaScript in Node.js, but we also made the WebAssembly module work with other languages and environments such as Java, PHP, Python, Rust and browsers. The interoperability between the other languages and the WebAssembly module works in the same way as the above described steps with JavaScript. Except that we used the Wasmer runtime, since those other languages don’t natively support WebAssembly.
Example
If you want to check out the Proof of Concept, see the following repository. While we initially made the Proof of Concept as simple as possible, we later improved it by adding partials support and refactored the code of the languages that was needed to interop with the WebAssembly module into classes, to abstract away the details.
Next step: Integration of a JavaScript engine
The Proof of Concept validated our vision and gave us insight into what is possible with WebAssembly. Having said that, we already started working on the next step: integration of a JavaScript engine into a WebAssembly module. This is a game changer, since we can run (unsafe) JavaScript on back-ends with different tech stacks. There wouldn’t be a need for Rust specific libraries like Handlebars-rust to be integrated in WebAssembly modules anymore. We could for example pass the JavaScript implementation of Handlebars to the engine and subsequently render templates with it.
Unfortunately the engine (see Boa) that we integrated into a WebAssembly module currently covers around 75% of Test262, the ECMAScript conformance test suite. Making pretty much any JavaScript library unusable unless modified. Nonetheless, the engine will improve in time.
