WASM: The `What`, `When` and `How`

I’ve been using WebAssembly aka WASM for a while in production to do some incredible stuff in the browser, things that would be prohibitive in terms of performance otherwise. Here are some real use-cases I’ve used WASM for:

1. Running statistical processing 10 million rows (roughly 50 columns in each row) of CSV data in browser. This feature required us to create a temporary playground of reports generated for our clients where they could run their own analysis without the need of permanent storage or costly servers.
2. Parsing DOCX and PPTX files in browser and running DL Models on the data in browser. We saved thousands of $ by eliminating the persistent backend and reduced the time to result from minutes to a few seconds.
3. One use-case ended up having tons of complex user input validations, we were building these validators on the server-side anyway, and with WASM we just put the same validator code in the client improving productivity and minimizing chances of mistakes and discrepancies between client and server validators.
4. Audio-Video encoding in browser, saving time and thus making content generation more dynamic and open to experimentation.

And a few more …

Since you are here you’ve probably already heard of or read about the success stories of Figma, Adobe has deployed their browser based Acrobat, Lightroom etc. using WASM, Google uses WASM in countless products and plenty more. WASM also finds it’s uses beyond the browser - Shopify Functions, Cloudflare Workers and many more.

In this post, I’m going to focus on WASM in Browser by building a small text-diff application with Rust.

TL; DR

Github Repo for this project

The Final Output

What is WASM?

I think Mozilla provides us with the best introductory definition:

WebAssembly

WebAssembly is a type of code that can be run in modern web browsers — it is a low-level assembly-like language with a compact binary format that runs with near-native performance and provides languages such as C/C++, C# and Rust with a compilation target so that they can run on the web. It is also designed to run alongside JavaScript, allowing both to work together.

-— MDN

And my nutshell version

WASM brings the power of low-level programming languages to the browser environment. That’s it!

When to use WASM?

It’s important to note that WASM is not a replacement for your current web app stack. JavaScript, HTML, CSS are the native standards, and they are your primary building blocks for your web-app. Think of WASM as a tool to augment your browser stack or a tool to eliminate bottlenecks.

Here are my top 3 reasons of using WASM:

1. Performance: If your application’s Data Processing far outweighs DOM Manipulation, WASM will help.
2. Portability: If you use a bunch of battle-tested libraries but their JS port is not available, not as performant or not feature-complete, WASM might help
3. Edge Compute: This has been the most common trigger for my WASM usage. If your application does a bunch of stuff that would require backend-level programs, but the client-server communication latency is a major limiting factor to the experience; or a specific feature is not used frequently enough to justify a backend server deployment - WASM probably would be the right tool for you.

How to use WASM?

Ok, enough with the rationale and explanations let’s dive into building something real. We are going to be creating a simple web-application to display a diff between two text blocks and hopefully in this process see some performance benefits. We are going to write a simple Rust program to do the diffing in WASM while rest of our interface will be in HTML + JS.

Setup

Let’s start by creating a basic Webpack project.

mkdir wasm-diff
cd wasm-diff
npm init -y
npm install --save-dev webpack webpack-cli webpack-dev-server html-webpack-plugin nodemon
npm i -D tailwindcss
npm i -D style-loader css-loader postcss postcss-loader postcss-preset-env
npx tailwindcss init

In the first 3 commands we initialized a basic npm project, the 4th line is webpack related stuff. The 5th, 6th and 7th lines are for tailwindcss library, I just HATE working with vanilla CSS.

Configuring Tailwind

Follow these instructions for configuring tailwindcss for our project. This is optional and vanilla CSS will work just fine.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56

<!DOCTYPE html>
<html>
    <!-- Some meta stuff, ommitted for readability -->
    <head><script defer src="bundle.js"></script></head>
    <body>
        <div class="canvas">
            <h1 class="text-4xl leading-loose">WASM Diff</h1>
            <div class="grid grid-cols-2 gap-2">
                <div class="relative flex flex-col gap-2">
                    <label for="old" class="block mb-2 text-sm font-medium">Old Text</label>
                    <textarea rows="24" id="old-textarea" class="textarea"></textarea>
                </div>
                <div class="relative flex flex-col gap-2">
                    <label for="new" class="block mb-2 text-sm font-medium">New Text</label>
                    <textarea rows="24" id="new-textarea" class="textarea"></textarea>
                </div>
            </div>
            <div class="grid grid-cols-2 gap-2 py-4">
                <div class="flex flex-col">
                    <div class="flex flex-row">
                        <h2 class="text-2xl">JavaScript Diff</h2>
                        <div class="ml-auto">
                            <button type="button" class="button purple" id="button-js-diff">
                                Run JS Diff
                            </button>
                        </div>
                    </div>
                    <div class="stats">
                        <div class="stat-sect">
                            <label for="js-total-time">Total Time</label>
                            <div class="stat-data" id="js-total-time">-</div>
                        </div>
                    </div>
                    <div class="diff" id="show-js-diff"></div>
                </div>
                <div class="flex flex-col">
                    <div class="flex flex-row">
                        <h2 class="text-2xl">WASM Diff</h2>
                        <div class="ml-auto">
                            <button type="button" class="button blue" id="button-wasm-diff">
                                Run WASM Diff
                            </button>
                        </div>
                    </div>
                    <div class="stats">
                        <div class="stat-sect">
                            <label for="ws-total-time">Total Time</label>
                            <div class="stat-data" id="ws-total-time">-</div>
                        </div>
                    </div>
                    <div class="diff" id="show-wasm-diff"></div>
                </div>
            </div>
        </div>
    </body>
</html>

Now, let’s wrap up our working app with a working src/index.js file.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

 // tailwind and modern css processor magic
import "./style.css";
 // keeping text data separate - for readability in `src/data.js` file
import { NEW_TXT, OLD_TEXT } from "data";

let oldtxtarea, newtxtarea;
let jsdiffbutton, wasmdiffbutton;

const jsDiff = () => {
    alert("JS Diff");
}

const wasmDiff = () => {
    alert("Wasm Diff");
}

// Once window is loaded initializing the event listeners and binding to specific DOM elements
window.onload = () => {
    // Binding the textarea elements
    oldtxtarea = document.getElementById("old-textarea");
    newtxtarea = document.getElementById("new-textarea");

    // Binding the buttons
    jsdiffbutton = document.getElementById("button-js-diff");
    wasmdiffbutton = document.getElementById("button-wasm-diff");

    // For our demo we are pre-populating the largish text-data here
    if(oldtxtarea && newtxtarea) {
        oldtxtarea.value = OLD_TEXT;
        newtxtarea.value = NEW_TXT;
    }

    // your basic event listeners
    jsdiffbutton.onclick = jsDiff;
    wasmdiffbutton.onclick = wasmDiff;
}

Let us run our app now to see if everything is in order.

npm run dev

Navigate to http://localhost:8080 in your browser and you should see the app running.

That should complete our setup! Let’s move on to getting the diff to work.

Diff with JavaScript

We are going to be using diff-match-patch npm package for our JavaScript diff, let’s install it.

npm i diff-match-patch

The interface is simple, we’ll use the diff_main API of the package to create the diffs of our old and new texts and then use the example diff_prettyHtml API to create the HTML string and display it.

Let’s change our src/index.js to do that.

 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49

// Initializing the JS diff class
const jsdmp = new DiffMatchPatch();

// A simple postprocessing function to replace existing styles to suit our cause
// Instead of calling `dmp.diff_prettyHtml()` we should ideally have our own function to format the diff
const diffPrettier = (html) => {
    let cleaned = html.replaceAll("¶", "")
        .replaceAll(" style=\"background:#ffe6e6;\"", "")
        .replaceAll(" style=\"background:#e6ffe6;\"", "");

    return cleaned;
}

// Calculate diff with the JS module
const jsDiff = () => {
    let old_txt = oldtxtarea.value;
    let new_txt = newtxtarea.value;

    // Capturing start time for stats
    let start = Date.now();
    // create our diffs
    let diffs = jsdmp.diff_main(old_txt, new_txt, true);
    // this API makes diff more human readable
    jsdmp.diff_cleanupSemantic(diffs);
    let html = jsdmp.diff_prettyHtml(diffs);

    let total = Date.now();

    html = diffPrettier(html);

    let target = document.getElementById("show-js-diff");
    let total_time = document.getElementById("js-total-time");
    if(!target || !total_time) {
        alert("Something went wrong! Reload the page");
        return;
    }

    target.innerHTML = html;
    total_time.innerText = `${(total - start)} ms`;
}

// .. code ommitted ..

We initialized the JavaScript DiffMatchPatch class, and in our jsDiff() function which is called by onclick event listener, we computed the diff, cleaned it up a bit and put the results and the stats in their respective DOM containers.

Note the diff time, in my current run it took about 512 ms to generate the diff. Let’s see if diffing with a WASM module adds any performance advantage to it or not!

We are going to compile a rust library to WASM, but if you are comfortable with C++, Go or C#, this will work just as well. Each one would have their own nuances to handle but that’s a story for another day!

Getting ready to WASM

Let’s start by creating a new rust project to house our WASM code, I’m creating an unimaginative wasm directory, it could be anything. We’ll use wasm-pack tool to help us with the build process.

Prerequisites

Rust toolchain

Assuming you have rust and cargo toolchain installed, let’s key in the commands.

curl https://rustwasm.github.io/wasm-pack/installer/init.sh -sSf | sh
mkdir wasm
cd wasm
cargo init --lib

The first command installs the cargo extension wasm-pack, through command 2 to 4 we are just creating a rust project of type --lib, it’ll not contain a main() function.

To ready our wasm crate let’s go ahead make some changes to wasm/Cargo.toml

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

[package]
name = "wasm"
version = "0.1.0"
edition = "2021"

[dependencies]
wasm-bindgen = "0"

# The `console_error_panic_hook` crate provides better debugging of panics by
# logging them with `console.error`. This is great for development, but requires
# all the `std::fmt` and `std::panicking` infrastructure, so isn't great for
# code size when deploying.
console_error_panic_hook = { version = "0" }

[lib]
crate-type = ["cdylib", "rlib"]

The wasm-bindgen crate we added to our [dependencies] section provides us with the batteries required to interface with JS and browser modules. The [lib] crate-type declaration basically tells our compiler to generate dynamic system library. This is used when compiling a dynamic library to be loaded from another language.

We’ll also need to modify our webpack.config.js to enable loading the generated wasm files.

 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

// .. code ommitted ...

module.exports = {
  entry: '< path to src/index.js >',
  mode: 'development',
  plugins: [
    // .. code ommitted ...
  ],
  output: {
    // .. code ommitted ...
  },
  module: {
    // .. code ommitted ...
  },
  experiments: {
    asyncWebAssembly: true
  }
};

By adding the declaration experiments.asyncWebAssembly: true we are telling webpack to load wasm modules asynchronously.

As a final step of the setup, let’s go ahead and add the build command to our package.json file to compile and generate the wasm modules and put it in our src/pkg directory.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

{
    "name": "wasm-diff",
    "version": "1.0.0",
    "description": "Text `diff` demo in browser with Rust WASM",
    "main": "index.js",
    "scripts": {
        "build": "webpack --config webpack.config.js",
        "dev": "wasm-pack build wasm --out-dir < path to src/pkg > --target web && webpack serve --config webpack.config.js"
    },
    // code ommitted
}

Let’s break down the dev command wasm-pack build wasm --out-dir ../src/pkg --target web && webpack serve --config webpack.config.js

• wasm-pack build - the cargo extension we installed earlier, we are using it to generate wasm modules from our rust library
• wasm - telling wasm-pack that our rust library code resides in <PROJECT_ROOT>/wasm directory
• --out-dir is telling wasm-pack to put the generated output files in <PROJECT_ROOT>/src/pkg directory
• --target web is telling wasm-pack to build for the browser. You could write a wasm library for say NodeJS environment, this option will change
• && webpack ... - rest of the command is basically to run the dev server we have already been running

That concludes our setup, if we run npm run dev now, we should see a bunch of files generated under the src/pkg directory.

But wait, nothing really happens yet! We have not written any code to call or do anything with the WASM module we just generated. Let’s work on that!

Hello World from WASM

We are going to write a function in our rust library return a Hello World string, and then we are going to call it from our JavaScript code.

Let’s modify our wasm/src/lib.rs file to return the Hello World.

1
2
3
4
5
6
7
8

use wasm_bindgen::prelude::*;

#[wasm_bindgen]
pub fn run() -> Result<String, String> {
    console_error_panic_hook::set_once();

    Ok("Hello from Wasm!".to_string())
}

The code is simple:

• the wasm-bindgen::prelude::* imports the batteries required to generate wasm modules and helpers that enable our rust fn or code to be called from JavaScript
• #[wasm_bindgen] macro tells the compiler that this function needs to be exposed for an external interface
• console_error_panic_hook::set_once(); is initializing a global hook so that if our wasm code generated from rust panics, a console.error would be thrown

Rest of it is basic rust code, just returning a Result<String>, Hello from Wasm! in this case.

Now, let’s add a couple of lines to our src/index.js file to call this function.

 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

// .. code ommitted ..
// importing our `wasm` module, the `init` initializer and `run` function we defined in our rust `wasm/src/lib.rs`
import init, { run } from "pkg";


let oldtxtarea, newtxtarea;
let jsdiffbutton, wasmdiffbutton;

// Initializing the JS diff class
const jsdmp = new DiffMatchPatch();

// Calling init asynchronously, on promise resolve we are logging the output
init().then(_ => {
    console.log(run());
});

// .. code ommitted ..

Basically we are importing the wasm initializer and initializing the wasm module, once the initializer resolves we are simply calling the run() function and printing its output.

Let’s run it .. if everything goes well, we should see Hello from Wasm! in our console!

npm run dev

And there you go, our first real stuff from WASM…

Diff with WASM

Let’s pick up a rust crate to do our Diff, there are a few options available if you search crates.io with the keywords diff match patch but as far as I know none of them support WASM out of the box. For this example, I’m going to use my own UNPUBLISHED crate diff-match-patch for the WASM diffing.

Pro Tips

Not all rust crates would be WASM compatible for various reasons. A sure-shot way of figuring this out is to navigate to docs.rs page for the crate and look through the Platforms tab. Most mature crates would show wasm32-unknown-unknown if they are WASM ready. The other way is to clone or use the crate in a toy project and use cargo check --target wasm32-unknown-unknown to look for errors. Some Crates might also enable WASM with feature flags.

Let’s add diff-match-patch-rs to our dependencies in wasm/Cargo.toml.

 4
 5
 6
 7
 8
 9
10

# .. code ommitted ..

[dependencies]
diff-match-patch-rs = { git = "https://github.com/anubhabb/diff-match-patch-rs.git", branch = "main" }
wasm-bindgen = "0"

# .. code ommitted ..

Now, let’s change our wasm/src/lib.rs to do the actual diffing.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

use diff_match_patch_rs::dmp::DiffMatchPatch;
use wasm_bindgen::prelude::*;

#[wasm_bindgen]
#[derive(Default)]
pub struct Differ {
    dmp: DiffMatchPatch,
}

#[wasm_bindgen]
impl Differ {
    #[wasm_bindgen(constructor)]
    pub fn new() -> Self {
        console_error_panic_hook::set_once();
        
        let dmp = DiffMatchPatch::default();
        Self {
            dmp
        }
    }

    pub fn diff(&self, old: &str, new: &str) -> Result<String, String> {
        let diffs = match self.dmp.diff_main(old, new) {
            Ok(d) => d,
            Err(_) => {
                return Err("error while diffing".to_string())
            }
        };

        let html = match DiffMatchPatch::diff_pretty_html(&diffs[..]) {
            Ok(s) => s,
            Err(_) => {
                return Err("Error preparing HTML".to_string())
            }
        };

        Ok(html)
    }
}

Note

We don’t REALLY need to declare a separate struct Differ for our relatively simple use-case here, but often you’ll need to create an instance of a struct and access their methods. This is just to demonstrate the possibilities and introduce a few handy concepts.

Let’s break down what’s happening here.

• line 4-8 we are declaring a struct Differ and annotating it with the #[wasm_bindgen] macro, basically telling the compiler that instance of this struct will be accessed from foreign code.
• line 10-11 we are annotating the impl Differ block with #[wasm_bindgen] - in a more complex usecase we could have other impl Differ blocks which will not be accessed from our JavaScript, we’ll just skip #[wasm_bindgen] for those impl Differ blocks.
• line 12 is interesting, here we are using the #[wasm_bindgen(constructor)] macro to tell the compiler that the pub fn new() should be called as a constructor if we do new Differ() from our JavaScript code.

The rest of it serves the exact same flow to our JavaScript diff function call, it calls the diff_main() method of diff_match_patch_rs::dmp::DiffMatchPatch module instance and then created the html string and returns it.

Let’s modify our src/index.js to make use of this WASM diff code.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42

// .. code ommitted ..
// importing our `wasm` module, the `init` initializer and `Differ` struct `wasm/src/lib.rs`
import init, { Differ } from "pkg";

// code ommitted ...

// Initializing the WASM differ
let wsdmp;
init().then(_ => {
    wsdmp = new Differ();
});

// .. code ommitted ...

const wasmDiff = () => {
    if(!wsdmp) {
        console.error("Error initializing wasm Differ module");
        alert("WASM Differ was not initialized")
        return;
    }

    // Capturing start time for stats
    let start = Date.now();
    // create our diffs
    let html = wsdmp.diff(old_txt, new_txt, true);

    let total = Date.now();

    html = diffPrettier(html);

    let target = document.getElementById("show-ws-diff");
    let total_time = document.getElementById("ws-total-time");
    if(!target || !total_time) {
        alert("Something went wrong! Reload the page");
        return;
    }

    target.innerHTML = html;
    total_time.innerText = `${(total - start)} ms`;
}

// .. code ommitted ..

Let’s run it now …

npm run dev

And, then, lets browse to localhost:8080 and click the Run JS Diff and Run WASM Diff buttons …

🥁🥁🥁 … drum rolls …

WHOA, the WASM variant is 80% faster than the JavaScript version.

Closing thoughts & Next steps

80% speedup is a LOT of performance gain. In a static setup like our current version, it may not make a difference but imagine you are writing a collaborative editing tool like Google Docs, and you need to diff on every keyup … the performance gain we saw is going to be the difference between an usable and an unusable product.

Here are some ideas you can explore based on what you’ve done today:

1. WASM modules tend to be larger in size than your standard JavaScript variant. But good news is, there are ways of optimizing it. Read up about those optimizations (and easier ways of setting up WASM) in the rust wasm book. Here’s the chapter on specific size optimizations.

2. Modify the code and apply the patch to the source text. Hint: You’ll need to figure out a way of sending JsValue instead of plain String we’ve been using so far.

A word of caution

If you are using my diff-match-patch-rs crate, be careful. It’s still a work in progress and I’ll probably work on optimizations soon.

3. Convert this application to a real-time diff machine.

Before we close today …

If you have found this tutorial helpful consider spreading the word, dropping a star to the git repo or the crate repo it would act as a strong motivator for me to create more. If you found an issue or hit a snag, reach out to me @beingAnubhab. I’d love to hear about stuff you did with what you learnt today.

Acknowledgements, references and further reading

1. diff-match-patch is based on Myer's Diff algorithm, which has been a standard since 1989, how cool is that! This specific version we worked on is based on google/diff-match-patch library; the original source code can be found here.
2. We used wasm-bindgen which is a rust library and CLI tool that facilitates interactions between WASM modules and JavaScript code. Like many things in Rust, it’s well documented and there’s a book for that.
3. wasm-pack makes our life a lot easy when we work with Rust WASM and the book gives a nice and easy to follow walkthrough.
4. If you are as fascinated by WebAssembly as I am, and you are looking for the next cool project idea - browse through made with webassembly.