vim-wasm

npm package for vim.wasm

WARNING!: This npm package is experimental until v0.1.0 beta release.

Introduction

This is an npm package to install pre-built vim.wasm binary easily. This package contains:

• vim.wasm: WebAssembly binary
• vim.js: Web Worker script to drive vim.wasm
• vim.data: Bundled preloaded files loaded into filesystem at start up
• vimwasm.js: ES Module to manage lifetime of Web Worker
• small/vim.{wasm,js,data}: Small feature version

Please read the following instructions to use this package. You can play with live demo. For usage of the demo, please read usage documentation.

Installation

Install npm package via npm command.

npm install --save vim-wasm

Usage

NOTE: This npm package is currently dedicated for browsers. It does not work with Wasm interpreters outside browser like node.

Please see example directory for minimal live example and live demo for more complicated example.

Prepare index.html

Put <canvas> to render Vim screen and <input/> to take user input in HTML file.

<canvas id="vim-canvas"></canvas>
<input id="vim-input" autocomplete="off" autofocus />
<script type="module" src="index.js" />

Your script index.js must be loaded as type="module" because this npm package provides ES Module unless you use some JS source bundler.

Prepare index.js

import { VimWasm } from '/path/to/vim-wasm/vimwasm.js';
 
const vim = new VimWasm({
    canvas: document.getElementById('vim-canvas'),
    input: document.getElementById('vim-input'),
    workerScriptPath: '/path/to/vim-wasm/vim.js',
});
 
// Setup callbacks if you need...
 
// Start Vim (give option object if necessary)
vim.start();

VimWasm class is provided to manage Web Worker lifecycle where Vim is running. Please import it from vimwasm.js ES Module.

workerScriptPath is the most important option value which represents a file path to a worker script which runs Vim in Web Worker. By switching path to scripts vim-wasm/vim.js and vim-wasm/small/vim.js, you can switch feature set of Vim. Please read following 'Normal Feature and Small Feature' section.

VimWasm provides several callbacks to interact with Vim running in Web Worker. Please check example code for the callbacks setup.

Finally calling start() method starts Vim in new Web Worker. You can pass an options object to the method call to specify various options.

Serve index.html

Serve index.html with HTTP server and access to it from a web browser.

NOTE: This project uses SharedArrayBuffer and Atomics API. Only Chrome or Chromium-based browsers enable them by default. For Firefox and Safari, feature flag must be enabled manually for now to enable them. Please also read notices in README.md at the project page.

Related Projects

Following projects are related to this npm package and may be more suitable for your use case.

• react-vim-wasm: React component for vim.wasm. Vim editor can be embedded in your React web application.
• vimwasm-try-plugin: Command line tool to open vim.wasm including specified Vim plugin instantly. You can try Vim plugin without installing it!
• vim.wasm.ipynb: Jupyter Notebook integration with vim.wasm. Try it online!

Normal Feature and Small Feature

This package contains two binaries for 'normal' feature set and 'small' feature set. 'normal' feature set provides almost all Vim's powerful features but binary size is 4x bigger than 'small' feature set. 'small' feature set only provides basic features but binary size is much smaller.

	Normal Feature	Small Feature
Script path	vim-wasm/vim.js	vim-wasm/small/vim.js
Data size	1.3 MB	13 KB
Wasm size	709 KB	374 KB
Features	Almost all features	Not including syntax highlight, indentation, Vim script support, text objects, ...

Data size is significantly different because 'normal' feature set includes syntax highlighting and indentation support for all filetypes. In contrast 'small' feature set only includes colorscheme and minimal vimrc.

By passing a worker script path for each feature to workerScriptPath option, you can specify a feature set. Please choose one considering your application's requirements.

const normalVim = new VimWasm({
    canvas,
    input,
    workerScriptPath: '/path/to/vim-wasm/vim.js',
});
 
const smallVim = new VimWasm({
    canvas,
    input,
    workerScriptPath: '/path/to/vim-wasm/small/vim.js',
});

Check Browser Compatibility

This npm package runs Vim in Web Worker and the main thread communicates with the worker thread via SharedArrayBuffer. Chrome or Chromium based browser supports SharedArrayBuffer by default. Safari and Firefox supports it under a feature flag due to Spectre vulnerability.

To check current browser can use this package, checkBrowserCompatibility() utility function is provided. It returns an error message as string if it is not compatible (otherwise returns undefined).

const errmsg = checkBrowserCompatibility();
if (errmsg !== undefined) {
    alert(errmsg);
    return;
}
 
const vim = new VimWasm({...});

Debug Logging

Passing debug: true to VimWasm.start() method call enables debug logging with console.log.

vim.start({ debug: true });

Note: Debug logs in C sources are not controlled by the query parameter. It is controlled GUI_WASM_DEBUG preprocessor macro.

Performance Logging

Passing perf: true to VimWasm.start() method call enables the performance tracing. After Vim exits (e.g. :qall!), it dumps performance measurements in DevTools console as tables.

vim.start({ perf: true });

Note: For performance measurements, please ensure to use release build. Measuring with debug build does not make sense.

Note: Please do not use debug logging at the same time. Outputting console logs in DevTools slows application.

Program Arguments

Passing program arguments of vim command is supported.

Passing cmdArgs option to VimWasm.start() method call passes the value as Vim command arguments.

vim.start({
  cmdArgs: [ '~/.vim/vimrc', '-c', 'set number' ]
});

As shown in above example, this feature is useful when you want to open specific file at Vim startup.

Evaluate JavaScript with :!

:!/path/to/file.js

:! evaluates the JavaScript source file. In Vim's command line, % is replaced with the current buffer's file path so :!% evaluates the current buffer.

Currently only *.js files are executable via :! and any other commands are not supported. Console output is not captured. You need to open console in DevTools or use alert() to show value.

The JavaScript code is evaluated in main thread so DOM APIs are available. If the code throws an exception, the error message and stacktrace is shown as an error message in Vim.

Unlike :!, system() and systemlist() Vim script functions are explicitly not supported because their calls in Vim plugins are intended to execute shell commands.

Set Font

guifont option is available like other GUI Vim. All font names available in CSS are also available here. Note that only monospace fonts are considered. If you specify other font like serif, junks may remain on re-rendering a screen.

" Use 'Monaco' font
:set guifont=Monaco

If you want to specify font height also, :h{pixels} suffix is available.

" Use 'Monaco' font with 20px font height
:set guifont=Monaco:h20

Note that currently only font height can be specified in this format. And integer value is acceptable since Vim contains font size as integer internally.

If you want to specify font name and font height from JavaScript, set it via VimWasm.cmdline method.

vim.cmdline(`set guifont=${fontName}:h{fontHeight}`);

FileSystem Setup

VimWasm.start() method supports filesystem setup before starting Vim through dirs, files, fetchFiles and persistentDirs options.

dirs option creates new directories on filesystem. They are created on memory using emscripten's MEMFS by default. Note that nested directory paths are not available.

Notes:

• You must specify each parent directories to create a nested directory.
• Trying to create an existing directory causes an error.

// Create /work/documents directory
vim.start({
    dirs: ['/work', '/work/documents'],
});

persistentDirs option marks the directories are persistent. They are stored on Indexed DB thanks to emscripten's IDBFS. They will remain even if user closes a browser tab.

Notes:

• Marking non-existing directories as persistent causes an error. Please set dirs correctly to ensure they exists.
• Files are synchronized when Vim exits. Closing browser tab without :quit does not save files on Indexed DB. This behavior may change in the future.
• This option adds overhead to load files from database at Vim startup.

// Create /work/persistent directory. Contents of the directory are persistent
vim.start({
    dirs: ['/work', '/work/persistent'],
    persistentDirs: ['/work/persistent'],
});

files option creates files on filesystem. It is an object whose keys are file paths and values are files' contents as String.

Notes:

• Parent directories must exist for the files. If they don't exist, please create them by dirs option.
• You can overwrite default vimrc as below example.

// Create new file /work/hello.txt and overwrite vimrc
vim.start({
    dirs: ['/work'],
    files: {
        '/work/hello.txt': 'hello, world!\n',
        '/.vim/vimrc': 'set number\nset noexpandtab\n',
    },
});

fetchFiles option fetches remote resources and map them to filesystem entries. It is an object whose keys are file paths on filesystem and values are remote resource paths (relative file paths or URLs). It fetches file paths or URLs just before starting Vim and put them on filesystem.

vim.start({
    fetchFiles: {
        // Fetch hosted 'vim.js' file and put it as '/foo.js' in filesystem
        '/foo.js': '/vim.js',
        // Fetch 'README.md' file from remote with URL and put it as '/bar.md' in filesystem
        '/bar.md': 'https://raw.githubusercontent.com/rhysd/vim.wasm/wasm/README.md',
    },
});

By using this option, external files are easily loaded onto filesystem. For example, if you fetch plugin files, the plugin is available on Vim starting. vimwasm-try-plugin uses this option to load specified Vim plugin or colorscheme.

Resources (values of the object) are fetched with fetch(). Though all requests are sent asynchronously, Vim waits all responses. Fetching many files or too large file would slows Vim start up time so should be avoided.

Evaluate JavaScript from Vim script

To integrate JavaScript browser APIs into Vim script, jsevalfunc() Vim script function is implemented.

jsevalfunc({script} [, {args} [, {notify_only}]])

The first {script} argument is a string of JavaScript code which represents a function body. To return a value from JavaScript to Vim script, return statement is necessary. Arguments are accessible via arguments object in the code.

The second {args} optional argument is a list value which represents arguments passed to the JavaScript function. If it is omitted, the function will be called with no argument.

The third {notify_only} optional argument is a number or boolean value which indicates if returned value from the JavaScript function call is notified back to Vim script or not. If the value is truthy, function body and arguments are just notified to main thread and the returned value will never be notified back to Vim. In the case, jsevalfunc() call always returns 0 and doesn't wait the JavaScript function call has completed. If it is omitted, the default value is v:false. This flag is useful when the returned value is not necessary since returning a value from main thread to Vim in worker may take time to serialize and convert values.

The JavaScript code is evaluated in main thread as a JavaScript function. So DOM element and other Web APIs are available.

" Get Location object in JavaScript as dict
let location = jsevalfunc('return window.location')
 
" Get element text
let selector = '.description'
let text = jsevalfunc('
        \ const elem = document.querySelector(arguments[0]);
        \ if (elem === null) {
        \   return null;
        \ }
        \ return elem.textContent;
        \', [selector])
 
" Run script but does not wait for the script being completed
call jsevalfunc('document.title = arguments[0]', ['hello from Vim'], v:true)

Since values are passed by being encoded in JSON between Vim script, arguments passed to JavaScript function call and returned value from JavaScript function must be JSON serializable. As a special case, undefined is translated to v:none in Vim script.

" Error because funcref is not JSON serializable
call jsevalfunc('return "hello"', [function('empty')])
 
" Error because Function object is not JSON serializable
let f = jsevalfunc('return fetch')

The JavaScript function is called in asynchronous context. So await operator is available as follows:

let slug = 'rhysd/vim.wasm'
let repo = jsevalfunc('
        \ const res = await fetch("https://api.github.com/repos/" + arguments[0]);
        \ if (!res.ok) {
        \   return null;
        \ }
        \ return JSON.parse(await res.text());
        \ ', [slug])
echo repo

jsevalfunc() throws an exception when:

• some argument passed at 2nd argument is not JSON serializable
• JavaScript code causes syntax error
• evaluating JavaScript code throws an exception
• returned value from the function call is not JSON serializable

TypeScript Support

This npm package provides complete TypeScript support. Type definitions are put in vimwasm.d.ts and automatically referenced by TypeScript compiler.

Ported Vim

• Current version: 8.2.0055
• Current features: normal and small

Development

Sources

This directory contains a browser runtime for wasm GUI frontend written in TypeScript.

• pre.ts, runtime.ts: Runtime to interact with main thread and Vim on Wasm. It runs on Web Worker.
• main.ts, vimwasm.ts: Runtime to render a Vim screen and take user key inputs. It runs on main thread and is responsible for starting Web Worker.
• package.json: Toolchains for this frontend is managed by npm command. You can build this runtime by npm run build. You can run linters (eslint, stylelint) by npm run lint.

When you run ./build.sh from root of this repo, vim.wasm, vim.js, vim.data and main.js will be generated. Please host this directory on web server and access to index.html.

Files are formatted by prettier.

Testing

Unit tests are developed at test directory. Since vim.wasm assumes to be run on browsers, they are run on headless Chromium using karma test runner. Basically unit test cases run Vim worker and check draw events from it. Headless Chromium is installed in node_modules locally by puppeteer.

# Single run 
npm test
 
# Watch test source files and run tests on changes 
npm run karma
 
# Use normal Chromium with DevTools instead of headless version 
npm run karma -- --browsers ChromeDebug
 
# Apply eslint to TypeScript sources 
npm run lint
 
# Run visual testing. It checks Vim screen is rendered correctly 
npm run vtest

npm run lint and npm run vtest are run at git push by husky 🐶.

npm test is run at Travis CI for every remote push.

Notes

ES Modules in Worker

ES Modules and JS bundlers (e.g. parcel) are not available in worker because of emcc. emcc preprocesses input JavaScript source (here runtime.js). It parses the source but the parser only accepts specific format of JavaScript code. The preprocessor seems to ignore all declarations which don't appear in mergeInto call. Dynamic import is also not available for now.

• import statement is not available since the emcc JS parser cannot parse it
• Dynamic import is not available in dedicated worker: https://bugs.chromium.org/p/chromium/issues/detail?id=680046
• Bundled JS sources by bundlers such as parcel cannot be parsed by the emcc JS parser
• Compiling TS sources into one JS file using --outFile=xxx.js does not work since toplevel constants are ignored by the emcc JS parser

Offscreen Canvas

There were 3 trials but all were not available for now.

• Send <canvas/> to worker (runtime.js) by transferControlToOffscreen() and render draw events there. This is not available since runtime.js runs synchronously with Atomics API. It sleeps until main thread notifies. In this case, calling draw methods of OffscreenCanvas does nothing because rendering does not happen until JavaScript context ends (like busy loop in main thread prevents DOM rendering).
• In main thread, draw to OffscreenCanvas and transfer the rendered result to on-screen <canvas/> as ImageBitmap. I tried this but it was slower than simply drawing to <canvas/> directly. It is because sending rendered image to <canvas/> causes re-rending whole screen.
• Create another worker thread (renderer.js) in worker (runtime.js) and send draw events to renderer.js. In renderer.js, it renders them to OffscreenCanvas passed from main thread via runtime.js. This solution should be possible and is possibly faster than rendering draw events in main thread. However, it is currently not available due to Chromium bug https://bugs.chromium.org/p/chromium/issues/detail?id=977924. When a worker thread runs synchronously with Atomics API, new Worker instance cannot start because new worker is created in the same thread and starts asynchronously. This would be fixed by https://bugs.chromium.org/p/chromium/issues/detail?id=835717 but we need to wait for the fix.

License

Distributed under VIM License.

Example code is based on https://github.com/trekhleb/javascript-algorithms.

Copyright (c) 2018 Oleksii Trekhleb