Using JavaScriptCore with examples

in

Motivation

I like mixing languages and I especially like OCaml, JavaScript. Lately I’ve been writing OCaml bindings to JavaScriptCore, JSC is what underpins Safari both iOS and desktop variants, and ReactNative. When you’re writing bindings, you kinda need to know the C/C++ library that you’re binding to.

Example

We’re going to provide a pretty meaty example, we’ll make a custom object in C++ code, then we’ll use it from a JavaScript script. Imagine we want to expose a File like object, JS the language itself has no notion of a File, so it will need support from the C/C++ level; you can imagine this as the objects that say [native code].

Here’s the code, afterwards is a detailed breakdown (The R C++ feature confuses the syntax highlighter)

#include <string>
#include <iostream>
#include <JavaScriptCore/JavaScriptCore.h>

std::string getcwd_string(void)
{
   char buff[PATH_MAX];
   getcwd(buff, PATH_MAX);
   std::string cwd(buff);
   return cwd;
}

const char*
jsvalue_to_utf8_string(JSGlobalContextRef ctx, JSValueRef v)
{
  JSStringRef valueAsString = JSValueToStringCopy(ctx, v, NULL);
  size_t jsSize = JSStringGetMaximumUTF8CStringSize(valueAsString);
  char* jsBuffer = (char*)malloc(jsSize);
  JSStringGetUTF8CString(valueAsString, jsBuffer, jsSize);
  JSStringRelease(valueAsString);
  return jsBuffer;
}

void run_example(void)
{
  JSClassDefinition definition = kJSClassDefinitionEmpty;
  definition.className = "File";
  definition.callAsConstructor = [](auto ctx,
				    auto object,
				    auto argumentCount,
				    auto arguments[],
				    auto *exception) -> JSObjectRef {
    auto file_name =
    JSValueMakeString(ctx, JSStringCreateWithUTF8CString(getcwd_string().c_str()));
    auto prop_name = JSStringCreateWithUTF8CString("cwdName");
    auto example_obj = JSObjectMake(ctx, nullptr, nullptr);

    JSObjectSetProperty(ctx,
			example_obj,
			prop_name,
			file_name,
			kJSPropertyAttributeNone,
			nullptr);
    return example_obj;
  };

  auto ctx = JSGlobalContextCreate(nullptr);
  auto js_example_class = JSClassCreate(&definition);
  auto example_obj = JSObjectMake(ctx, js_example_class, nullptr);
  // auto called_result = JSObjectCallAsConstructor(ctx, example_obj, 0, nullptr, nullptr);
  // auto property_check =
  //   JSObjectGetProperty(ctx,
  // 			called_result,
  // 			JSStringCreateWithUTF8CString("cwdName"),
  // 			nullptr);

  // std::cout << jsvalue_to_utf8_string(ctx, property_check) << std::endl;
  std::string code_to_eval = R"(
const example_code = new File;
example_code.cwdName;
)";
  JSValueRef exn;

  auto global_object = JSContextGetGlobalObject(ctx);
  JSObjectSetProperty(ctx,
		      global_object,
		      JSStringCreateWithUTF8CString("File"),
		      example_obj,
		      kJSPropertyAttributeNone,
		      nullptr);

  auto sanity_check =
    JSEvaluateScript(ctx,
		     JSStringCreateWithUTF8CString(code_to_eval.c_str()),
		     nullptr,
		     nullptr,
		     1,
		     &exn);
  if (exn)
    std::cout << jsvalue_to_utf8_string(ctx, exn) << std::endl;

  std::cout << jsvalue_to_utf8_string(ctx, sanity_check) << std::endl;
}

int main(int argc, char **argv)
{
  run_example();
  return 0;
}

compile with, should work just fine on Linux as well, but using:

libjavascriptcoregtk-4.0 instead of the -framework JavaScriptCore

$ clang++ -framework JavaScriptCore -std=c++14 jsc_examples.cpp -o F

When you run ./F, then it should print out the current working directory.

Explanations

The real meat of the code starts at run_example. We start by creating a JSClassDefinition, this creates a template that lets us control all the behavior of our custom object. Then we provide an implementation for what code ought to be run when our custom object will created with new, notice using C++14’s nice ability to use auto for the parameter names, also notice that we explicitly have to give back the return type of JSObjectRef, you get a bit spoiled by the OCaml type system. Then we create a JavaScript object, set the property cwdName and return that new object from the constructor call.

Then we need to make a global context, this is like the environment that your JavaScript runs in and we grab the global object out of it, like window in the browser. (The commented out code is if you wanted to call the constructor and get the result directly in code)

We set the class object as a property to the global, aka like doing window.d3 = //d3's code. Then we evaluate a script, some fun trival all your <script> tags actually have a return value, its the last value.

Tada.

FWIW I’m doing the OCaml bindings here github stars appreciated :)

ReactJS, JSX, async/await, babel, webpack and getting it all working

in

Stranger in a strange land

I’m not primarily a front-end dev so getting everything setup and configured for web development is particularly frustrating for me and most blog posts, tutorials don’t actually give explanations, just copy-pasting blindly tons of little configs.

Here’s my post that I’m using as reference for me and hopefully for any other non-frontend developer that wants to use the latest and greatest JavaScript like fetch, async, await, React, JSX.

Getting started, compiling JavaScript to …JavaScript

Because of fragmentation in implementations of the latest JavaScript features, we’ll use babel to compile our JavaScript using the latest features to JavaScript that will work in Chrome, Firefox and Safari.

babel has a concept of plugins. These are like features that you can turn on during the compilation steps and are pretty granular. Often you’ll want a whole bunch of plugins together and that is so common enough that babel has something called presets. You can put that in separate .babelrc file, but I prefer not having so much silly little config files, so you can also put it in your package.json; example:

 "babel": {
    "presets": [
      "react",
      "es2015",
      "stage-3"
    ],
    "plugins": [
        "transform-es2015-modules-commonjs",
        "transform-async-to-generator",
        "transform-runtime"
    ]
  }

These are the ones I’m using to compile JSX, use ES6 modules, and async, await.

So when you invoke babel, it will look at the package.json, see the babel field and turn on those features, so an example invocation is:

$ babel lib --out-dir dist

which will compile all the code in the lib directory and output the results in the dist directory. This process is the same for node.

Bundling code

Now we have our legal JavaScript for today’s browsers/node. We can bundle up everything as a single JavaScript file using webpack. I previously used browserify but like all things web, apparently its not hot anymore. We can invoke it like so:

$ webpack --progress --colors dist/homepage.js bundle.js

where bundle.js is the name of the single output file that we’ll get. You can apparently do some kind of config file for webpack, yet another config file, but this is enough for me right now.

Actual code/project with JSX

So let’s say we have these two files, one is homepage.jsx and the other is button.jsx. Note that I use a real example of async, await, for a great explanation see here, for OCaml programmers, await is basically >>= or let%lwt.

This is button.jsx

'use strict';
import React from 'react';

class Button extends React.Component {

  async do_request(e) {
    let query =
	'https://api.bitcoinaverage.com' +
	'/ticker/global/USD';
    let nonsense = "https://foo.bar";
    try {
      let pulled = await fetch(query);
      let body = await pulled.json();
      console.log(body);

      await fetch(nonsense);

    } catch (e) {
      console.log("Exception raised:", e);
      console.log('Logic continued');
    }
  }

  render () {
    let s = {color:'red'};
    return (
      <p style={s}
	 onClick={this.do_request.bind(this)}>
	Click Me
      </p>
    );
  }
};
// Remember to put wrap in {}
export {Button};

and homepage.jsx

'use strict';

import React from 'react';
import ReactDOM from 'react-dom';
// REMEMBER to do {} since button.jsx doesn't do
// export default
import {Button} from './button';

class Page extends React.Component {
  render () {
    return (
      <div>
	Hello World
	<Button/>
      </div>
    );
  }
};

ReactDOM.render(<Page/>,
		document.getElementById('cont'));

So all this will be compiled correctly and turned into one bundle.js which we can use in this index.html

<!DOCTYPE html>
<meta charset="utf-8">
<body>
  <div id="cont"></div>
  <script src="bundle.js"></script>
</body>

and when we click the button we see this in the Chrome dev tools:

Object {24h_avg: 614.98, ask: 613.72, bid: 613.05, last: 613.56, timestamp: "Tue, 20 Sep 2016 20:09:30 -0000"…}
bundle.js:23085 GET https://foo.bar/ net::ERR_NAME_NOT_RESOLVED_callee$ @ bundle.js:23085tryCatch @ bundle.js:23242invoke @ bundle.js:23516prototype.(anonymous function) @ bundle.js:23275step @ bundle.js:23872(anonymous function) @ bundle.js:23883
bundle.js:23095 Exception raised: TypeError: Failed to fetch(…)
bundle.js:23096 Logic continued

Yay, things worked.

See the repo here for the full package.json

Calling a C function from within a binary on OS X/iOS

in

If you like this and other posts of mine, then reach out to me at edgar.factorial@gmail.com, I’m looking for remote contract work.

Let’s say we have a C binary given to us and there’s a C function in it that we want to call. For example, say we have this code, assume it is called inject_me.c:

#include <stdio.h>

int test_function(char *some_word)
{
  return printf("Did say: %s\n", some_word);
}

int main (int argc, char **argv)
{
  return test_function("Hello World");
}

and we want to call the function test_function by ourselves. To do that we need to find the location of the function in the binary. We can do that by disassembling the binary, I used to use command line tools for that but now I’m using hopper. (Hopper is an AMAZING program). Once we disassemble it, we find the function in memory:

Notice that I highlighted the actual location of: 0x100000f20. Now we can inject it with this code, assume it is called inject.c

#include <stdio.h>
#include <string.h>
#include <mach-o/dyld.h>

#define SPOT 0x100000f20

typedef int pull_it(char *);

static pull_it *pulled = NULL;

__attribute__((constructor))
void example_injection()
{
  char path[1024];
  uint32_t size = sizeof(path);
  _NSGetExecutablePath(path, &size);
  
  for (uint32_t i = 0; i < _dyld_image_count(); i++) {
    if (strcmp(_dyld_get_image_name(i), path) == 0) {
      intptr_t slide = _dyld_get_image_vmaddr_slide(i);

      pulled = (pull_it*)(intptr_t)(slide + SPOT);
      printf("slide: %lu\n", slide);
      pulled("Please work\n");
    }
  }

  printf("This ran before the actual program\n");
}
  1. The __attribute__((constructor)) calls the wrapped function before main goes off.

  2. The slide is needed because: (I found this on some Apple mailing list and don’t remember from where anymore)

Q: Can someone enlighten me as to what the virtual memory slide amount
is ? (this parameter is returned by _dyld_get_image_vmaddr_slide for
instance)

A: The shared libraries are prebound to an initial base address but
when the shared library gets loaded dyld can "slide" the library to a
new base address, this is where the virtual memory slide comes from.

Build both with this Makefile

osx_clang := $(shell xcrun --sdk macosx --find clang)
osx_sdk := $(shell xcrun --sdk macosx --show-sdk-path)
c_flags := -std=c11

inject_me:inject_me.c
	@${osx_clang} -isysroot ${osx_sdk} ${c_flags} $< -o $@

code_injection:inject_me
	@${osx_clang} -isysroot ${osx_sdk} ${c_flags} inject.c -dynamiclib -o $@.dylib

Now we can invoke it like so:

$ make code_injection
$ DYLD_INSERT_LIBRARIES=code_injection.dylib ./inject_me
slide: 125042688
Did say: Please work

This ran before the actual program
Did say: Hello World

Yay, it works. BTW, this works for jailbroken iOS as well.

Imperatively traverse binary trees in OCaml, print zigzag

in

OCaml tree examples tend to be defined with algebraic data types and tend to be functional examples. Here are two imperative tree traversals, a pre-order and in-order. I’m still trying to work out a nice post-order imperative solution in OCaml so if you have one then please tweet it at me: @edgararout

EDIT: I’ve added a zigzag function procedure as well. This question came up recently at an interview and I messed it up. The point is to print the tree in a zigzag pattern.

type 'a node = {mutable data: 'a;
                mutable left : 'a node option;
                mutable right: 'a node option; }

let new_node data = {data; left = None; right = None;}

let insert tree new_data =
  let module Wrapper = struct exception Stop_loop end in
  let iter = ref tree in
  try
    while true do
      if new_data < !iter.data
      then match !iter.left with
        | None ->
          !iter.left <- Some (new_node new_data);
          raise Wrapper.Stop_loop
        | Some left_tree -> iter := left_tree
      else if new_data > !iter.data
      then match !iter.right with
        | None ->
          !iter.right <- Some (new_node new_data);
          raise Wrapper.Stop_loop
        | Some right_tree -> iter := right_tree
    done
  with Wrapper.Stop_loop -> ()

let pre_order_traversal tree =
  let s = Stack.create () in
  Stack.push tree s;
  while not (Stack.is_empty s) do
    let iter_node = Stack.pop s in

    Printf.sprintf "%s " iter_node.data
    |> print_string;

    (match iter_node.right with
       None -> ()
     | Some right -> Stack.push right s);

    (match iter_node.left with
       None -> ()
     | Some left -> Stack.push left s)
  done

let in_order_traversal tree =
  let module W = struct exception Stop_loop end in
  let visited_stack = Stack.create () in
  let iter_node = ref (Some tree) in
  try while true do
      (* Inner loop, we keep trying to go left *)
      (try while true do
           match !iter_node with
           | None -> raise W.Stop_loop
           | Some left ->
             Stack.push left visited_stack;
             iter_node := left.left
         done;
       with W.Stop_loop -> ());

      (* If we have no more to process in the stack, then we're
         done *)
      if Stack.length visited_stack = 0
      then raise W.Stop_loop
      else
        (* Here we're forced to start moving rightward *)
        let temp = Stack.pop visited_stack in
        Printf.sprintf "%s " temp.data |> print_string;
        iter_node := temp.right
    done
  with W.Stop_loop -> ()

let print_spiral root =
  let (current, next) = Stack.(ref (create ()), ref (create ())) in
  let left_to_right = ref true in

  let swap a b = let (a_, b_) = !a, !b in a := b_; b := a_ in

  Stack.push root !current;

  while not (Stack.is_empty !current) do
    let r = Stack.top !current in
    Stack.pop !current |> ignore;
    Printf.sprintf "%s " r.data |> print_string;
    if !left_to_right then
      begin
        (match r.left with None -> () | Some l -> Stack.push l !next);
        (match r.right with None -> () | Some r -> Stack.push r !next)
      end
    else begin
      (match r.right with None -> () | Some r -> Stack.push r !next);
      (match r.left with None -> () | Some l -> Stack.push l !next)
    end;

    if Stack.length !current = 0
    then (left_to_right := not !left_to_right; swap current next)

  done

let () =
  let root = new_node "F" in

  ["B";"G";"A";"D";"I";"C";"E";"H"] |> List.iter (insert root);

  pre_order_traversal root;
  print_newline ();
  in_order_traversal root
  print_newline ();
  print_spiral root

Intermediate js_of_ocaml

in

This post is mainly aimed at intermediate level users of js_of_ocaml, the OCaml to JavaScript compiler and at improving the sad state of documentation for js_of_ocaml. (Plus I’m interviewing for jobs, hire me!, and wanted to use both OCaml, JavaScript while preparing for interviews)

OCaml typing of JavaScript (Uses PPX)

Say we want to express some algorithms in OCaml but using JavaScript as the runtime execution language/environment. Let’s start with our main data structure, a tree node.

function TreeNode(value) {
  this.value = value;
  this.left = this.right = null;
}

Our OCaml version will be:

class type ['data] tree_node = object
  method value : 'data Js.prop
  method left : 'data tree_node Js.t Js.prop
  method right : 'data tree_node Js.t Js.prop
end

Note that there is a small thing about typing the nullability of the left, right fields, I will return to this at the end of the blog post and it was done for convenience.

So explanations:

  1. We are creating a class type, this just describes the object, how we describe it is key.
  2. The 'data is a type variable and lets us use any kind of type for the value in this node.
  3. OCaml objects only expose methods to the outside world, so properties need Js.prop, this lets us read and write to this field. We can control it to be read only by instead using Js.readonly_prop or even write only with Js.writeonly_prop.
  4. Methods left, right are also properties. Read the signature from right to left, aka left is a JavaScript read or write property which has a JavaScript object typed as tree_node and parameterized with the 'data type variable. Aka tree_nodes of string or int or whatever the type of 'data is.

Now we need to provide a way to make this object.

let __hidden__ =
  Js.Unsafe.pure_js_expr "function TreeNode(value) {\
                          this.value = value; \
                          this.left = this.right = null;}"

This is the JavaScript we’ll be using, essentially. Do note that the field names match up, this is important. Now we provide a constructor function:

let node : ('data -> 'data tree_node Js.t) Js.constr = __hidden__

This node constructor says that its a special JavaScript constructor that will invoke __hidden__ with new and such that it expects one argument. An example is:

let root = new%js node "Hello"

We can also make specialized constructors,

let node_from_int : (int -> int tree_node Js.t) Js.constr = __hidden__

and a simple usage:

let () =
  let root = new%js node "Hello" in
  root##.left := new%js node "Left side";
  root##.left##.left := new%js node "Grand Kid";

  (* Note that this just prints the raw object representation of the
  field value *)
  Firebug.console##log root##.left##.left##.value;

  (* This shows the value as expected *)
  print_endline root##.left##.left##.value

You can compile and run it on node with: (Assuming file name is trees_in_js.ml)

$ ocamlfind ocamlc -package js_of_ocaml.ppx -linkpkg trees_in_js.ml
$ js_of_ocaml a.out -o T.js
$ node T.js

And you should get something like the following printed out:

h { t: 0, c: 'Grand Kid', l: 9 }
Grand Kid

Depth First Search, Level order traversal complete examples

Now here’s an example of a preorder depth first search and level order traversal.

class type ['data] tree_node = object
  method value : 'data Js.prop
  method left : 'data tree_node Js.t Js.prop
  method right : 'data tree_node Js.t Js.prop
end

let node : ('data -> 'data tree_node Js.t) Js.constr = __hidden__

let depth_first_search starting_node =
  let stack = Stack.create () in
  Stack.push starting_node stack;
  while not (Stack.is_empty stack) do
    let iter_node = Stack.pop stack in

    Printf.sprintf "%s " iter_node##.value
    |> print_string;

    if Js.Opt.return iter_node##.right |> Js.Opt.test
    then Stack.push iter_node##.right stack;

    if Js.Opt.return iter_node##.left |> Js.Opt.test
    then Stack.push iter_node##.left stack

  done

let level_order starting_node =
  let q = Queue.create () in
  Queue.add starting_node q;

  while not (Queue.is_empty q) do
    let pop = Queue.pop q in
    Printf.sprintf "%s " pop##.value
    |> print_string;

    if Js.Opt.(return pop##.left |> test)
    then Queue.push pop##.left q;

    if Js.Opt.(return pop##.right |> test)
    then Queue.push pop##.right q

  done

let () =
  let root = new%js node "F" in
  root##.left := new%js node "B";
  root##.right := new%js node "G";
  root##.right##.right := new%js node "I";
  root##.left##.left := new%js node "A";
  root##.left##.right := new%js node "D";
  root##.left##.right##.left := new%js node "C";
  root##.left##.right##.right := new%js node "E";
  root##.right##.right##.left := new%js node "H";

  depth_first_search root |> print_newline;
  level_order root |> print_newline

and compile it just like given earlier in the blog post. Shameless plug, you can also turn it into an executable with a feature I added to the js_of_ocaml compiler,

$ ocamlfind ocamlc -package js_of_ocaml.ppx -linkpkg trees_in_js.ml
$ js_of_ocaml --custom-header='#!/usr/bin/env node' a.out -o T.js
$ chmod +x T.js
$ /T.js

Yay, we used the resources of two programming languages standard libaries in one program!

Now we can return to why the typing of the class type matters. The fully correct typing of tree_node is:

class type ['data] tree_node = object
  method value : 'data Js.prop
  method left : 'data tree_node Js.t Js.Opt.t Js.prop
  method right : 'data tree_node Js.t Js.Opt.t Js.prop
end

Notice the addition of Js.Opt.t. Since the left, right are nullable, we should capture that in the OCaml API, this however does force us to use Js.Opt and so we’d have to do things like:

root##.left := new%js node "Left side" |> Js.Opt.return;

When trying to set the field, etc. But since we didn’t expose that nullability of the field in the type signature, the depth_first_search code needs to check if the field is indeed null, which would have been otherwise forced by the type system had we used Js.Opt, aka:

if Js.Opt.return iter_node##.right |> Js.Opt.test
then Stack.push iter_node##.right stack;

These are tradeoffs that you can make in your own usage.

I hope this makes it easier for you to use OCaml, JavaScript together.