Preliminary Performance Tests for mruby and V8

mruby 2 Comments
Note: We changed the name of the database in May 2012. AvocadoDB is now called ArangoDB

I’m still investigating the possibility to use mruby as embedded language for AvocadoDB, see me last post. I managed to create an interactive shell to play with mruby. Now am trying to do some performance tests.

Note that mruby is still “alpha”, so DO NOT take these test to seriously. They are basically just for me to learn, how to use mruby.

Exposing a C function to MRUBY

The first function I need is a timing function. I’ve implemented such a function in C using gettimeofday. Now in order to make it accessible from mruby I’ve used

static mrb_value MR_Time (mrb_state* mrb, mrb_value self) {
  return mrb_float_value(TRI_microtime());
}
 
void TRI_InitMRUtils (mrb_state* mrb) {
  struct RClass *krn;
 
  krn = mrb->kernel_module;
 
  // timing function
  mrb_define_method(mrb, krn, "time", MR_Time, ARGS_NONE());
}

mrb_define_method defines a new method in a class or module. I’m not sure what the correct module or class is, so for my tests I’ve chosen the Kernel module. ARGS_NONE() is used for functions that take no arguments.

MRuby

Testing it as follows works

fceller@huerth:~/Arango-DB-MRUBY> ./avocirb 
                        _      _      
   __ ___   _____   ___(_)_ __| |__   
  / _` \ \ / / _ \ / __| | '__| '_ \  
 | (_| |\ V / (_) | (__| | |  | |_) | 
  \__,_| \_/ \___/ \___|_|_|  |_.__/  
 
Welcome to avocirb 0.3.13. Copyright (c) 2012 triAGENS GmbH.
Using MRUBY 0.0.0 engine. Copyright (c) 2012 mruby developers.
Using READLINE 6.1.
 
avocirb> def fact n; (n > 1) ? n*fact(n-1) : 1; end
avocirb> $a = time; $b = fact(100); p time - $a
9.08374786376953e-05

As expected it is too fast to be usable. So, I’ve used a loop

avocirb> $a = time; for i in 1 .. 10000000; fact(10.0); end; p time - $a
20.9153339862823
avocirb> $a = time; for i in 1 .. 10000000; $b = fact(10.0); end; p time - $a
21.3838360309601
avocirb> p $b
3628800.0

So there is also no big difference between integer and floats. Both of with are stored as immediate values in mruby. Note that there is no Bignum in mruby.

JavaScript

The same in JavaScript V8 give:

                            _         
   __ ___   _____   ___ ___| |__      
  / _` \ \ / / _ \ / __/ __| '_ \   
 | (_| |\ V / (_) | (__\__ \ | | | 
  \__,_| \_/ \___/ \___|___/_| |_|   
 
Welcome to avocsh 0.3.13. Copyright (c) 2012 triAGENS GmbH.
Using Google V8 3.9.4 JavaScript engine.
Using READLINE 6.1.
 
avocsh> function fact (n) { return 1 < n ? n * fact(n-1) : 1; }
avocsh> a = internal.time(); for (var i = 0;  i < 10000000;  ++i) { b = fact(10); } print(internal.time() - a);
0.6311039924621582
avocsh> a = internal.time(); for (var i = 0;  i < 10000000;  ++i) { b = fact(10.0); } print(internal.time() - a);
0.6052999496459961
avocsh> print(b);
3628800

JavaScript is currently faster – but note that mruby is still in an very early stage. BTW, I’ve used “-O2″ for both V8 and mruby.

Ruby 1.8.7

Using “irb” one gets

irb(main):001:0> def fact n; (n > 1) ? n*fact(n-1) : 1; end
=> nil
irb(main):011:0> $a = Time.now; for i in 1 .. 10000000; $b = fact(10); end; p Time.now - $a
49.044638
=> nil
irb(main):010:0> $a = Time.now; for i in 1 .. 10000000; $b = fact(10.0); end; p Time.now - $a
61.320039

So in this particular example, mruby is a lot faster than Ruby 1.8. Again, it is too early to draw any conclusions.

Ruby 1.9

Using “irb” one gets

irb(main):001:0> def fact n; (n > 1) ? n*fact(n-1) : 1; end
=> nil
irb(main):002:0> $a = Time.now; for i in 1 .. 10000000; $b = fact(10); end; p Time.now - $a
7.81386388
=> 7.81386388
irb(main):003:0> $a = Time.now; for i in 1 .. 10000000; $b = fact(10.0); end; p Time.now - $a
22.378524013
=> 22.378524013

Next Steps

I need to do more tests to see how easy it is to really embed the language in to ArangoDB. And I also need to do some more tests.

About Frank Celler

Frank is both entrepreneur and backend developer, developing mostly memory databases for two decades. He is the lead developer of ArangoDB and co-founder of triAGENS. Try to challenge Frank asking him questions on C, C++ and MRuby. Besides Frank organizes Cologne’s nosql group & nosql conferences.
  • jan steemann

    I was interested in whether the huge performance difference you measured was due to MRuby being (probably) still unoptimized or due to V8 being much faster than other dynamic language executors. So I measured myself.
    Results from a 32 bit laptop follow:

    MRuby (0.0.0):

    avocirb>  def fact n; (n > 1) ? n*fact(n-1) : 1; end
    avocirb> $a = time; for i in 1 .. 10000000; fact(10.0); end; p time – $a
    57.8161611557007
    avocirb> $a = time; for i in 1 .. 10000000; $b = fact(10.0); end; p time – $a
    59.1052560806274
    avocirb> p $b
    3628800.0

    Javascript (V8 3.9.4):

    avocado> function fact (n) { return 1 a = internal.time(); for (var i = 0;  i a = internal.time(); for (var i = 0;  i print(b);
    3628800

    The results are in line with what you measured. V8 was about the same magnitude faster than MRuby as in your test.
    For curiosity I also cross checked the results against PHP.

    PHP (5.3.6):

    cli> php -r ‘function fact($n) { return ($n > 1) ? $n * fact($n – 1) : $n; } $a = microtime(true); for ($i = 1; $i php -r ‘function fact($n) { return ($n > 1) ? $n * fact($n – 1) : $n; } $a = microtime(true); for ($i = 1; $i < 10000000; $i++) { $b = fact(10.0); } print microtime(true) – $a; print "n"; print $b; print "n";'
    36.732476949692
    3628800

    So MRuby and PHP execution times at least have the same order of magnitude, which was what I expected because I think they have the same code execution model.

    • cabo

      Comparing Ruby1.9 and MacRuby (another JITted Ruby) on my MBA (which is slightly slower than the other machines used here):

      ruby1.9 ~/tmp/ben1.rb11.44724430.074118macruby ~/tmp/ben1.rb4.6390053.818616Obviously the unboxed floating point values of MacRuby seriously win, but also the JIT seems to win (but not as much as V8, unsurprisingly).V8 version 2.2.4.2> function fact (n) { return 1 a = new Date; for (var i = 0;  i a = new Date; for (var i = 0;  i  (these are milliseconds).