Warning Older Docs! - You are viewing documentation for a previous released version of RhoMobile Suite.

RhoConnect Benchmarks

Note, these numbers reflect synchronization performance using the rhoconnect bench utilities (see the rhoconnect/bench folder for more information).
We will be adding more results as we test across different server environments.

Single-core Amazon EC2 server environment
Multi-core Amazon EC2 server environments

Single-core Amazon EC2 server environment

Configuration

Servers configuration (Amazon EC2 ‘m1.small’ instances) :

CPU Units - 1 ECU
CPU Cores - 1 Core
Memory - 1.7GB
Platform - Ubuntu (2.6.38-8-virtual)

Environment (instance #1):

Nginx 1.0.0
Ruby 1.8.7 (2011-02-18 patchlevel 334) [i686-linux], Ruby Enterprise Edition 2011.03
Passenger 3.0.7 (mod_rails)
Redis 2.2.11
Rhoconnect 3.0.0

Nginx configuration file:

user  www-data;
worker_processes  4;

pid /var/run/nginx.pid;

events {
    worker_connections  1024;
}

http {
    passenger_root /usr/local/lib/ruby/gems/1.8/gems/passenger-3.0.7;
    passenger_ruby /usr/local/bin/ruby;
    passenger_max_pool_size 20;
    include       mime.types;
    default_type  application/octet-stream;

    sendfile    on;
    tcp_nopush  on;
    tcp_nodelay on;

    keepalive_timeout  65;

    gzip  on;

    server {
        listen 80;
        server_name  ec2-50-18-72-186.us-west-1.compute.amazonaws.com;
        root  /var/www/benchapp/public;   # Bench application #1
        # root  /var/www/blobapp/public;   # Bench application #2
        passenger_enabled on;
        passenger_min_instances 6;
    }
}

Environment (instance #2):

Ruby 1.8.7 (2011-02-18 patchlevel 334) [i686-linux], Ruby Enterprise Edition 2011.03
Rhoconnect 3.0.0 code with bench scripts

Server instance #1 is running rhoconnect bench applications, the 2nd one executes rhoconnect bench scripts.

Bench application #1 (benchapp)

Simulates creating multiple objects
100 concurrent users
1 device per user
20 object dataset
each object has 7 attributes
904 total HTTP requests

Request clientcreate   : min: 0.0069, max: 1.6010, avg: 0.3404
Request ack-cud        : min: 0.0339, max: 3.2250, avg: 0.5910
Request clientlogin    : min: 0.0063, max: 0.8138, avg: 0.1216
Request get-cud        : min: 0.0315, max: 3.0044, avg: 0.3988
Request create-object  : min: 0.0791, max: 4.4959, avg: 1.3521
Throughput(req/s)      : 7.13629650622702
Throughput(req/min)    : 428.177790373621

Bench application #2 (blobapp)

Simulates creating multiple objects
100 concurrent users
1 device per user
10 object dataset
each object has variable number of attributes (from 10 to 60) + image
904 total HTTP requests

Request clientcreate   : min: 0.0067, max: 2.9286, avg: 0.6870
Request ack-cud        : min: 0.0473, max: 5.5015, avg: 1.6219
Request clientlogin    : min: 0.0061, max: 3.2326, avg: 0.4687
Request get-cud        : min: 0.0382, max: 7.2202, avg: 1.0975
Request create-object  : min: 0.3765, max: 11.1387, avg: 5.1424
Throughput(req/s)  : 7.22833609137099
Throughput(req/min): 433.700165482259

Prepare bench applications for running on server #1

Go to rhoconnect/bench directory and configure benchapp application:

sudo cp -R benchapp /var/www/
cd /var/www/
sudo mkdir benchapp/public
sudo chown -R www-data:www-data benchapp/

Do the same for blobapp bench:

sudo cp -R blobapp /var/www/
cd /var/www/
sudo mkdir blobapp/public
sudo chown -R www-data:www-data blobapp/

Edit Nginx configuration file, set server_name and root properties to public DNS server name and bench application public folder respectively. After restarting server make sure that bench application is running by visiting its web console, i.e. :

http://server_dns_public_name/console/

How to execute bench scripts on server #2

Before running bench scripts you need configure bench application settings properly.

Edit rhoconnect/bench/benchapp/settings/settings.yml and set redis and syncserver properties:

:development:
  :licensefile: settings/license.key
  :redis: server_ip_address:6379
  :syncserver: http://server_dns_public_name:80/api/application/

Do the same for blobapp settings.yml file.

Then from rhoconnect/bench directory execute run_cud_script.sh and run_blob_script.sh scripts respectively.

Multi-core Amazon EC2 server environments

Configuration

Server configurations:

Large (m1.large, 2 cores, 7.5GB memory)
Extra Large (m1.xlarge, 4 cores, 15GB memory)
High-CPU Extra Large (c1.xlarge, 8 cores, 7GB memory)
Platform - Basic 64-bit Amazon Linux AMI 2011.09 (Red Hat based)

Software environment:

Nginx 1.0.6
Ruby 1.8.7 (2011-02-18 patchlevel 334) [i686-linux], Ruby Enterprise Edition 2011.03
Passenger 3.0.9
Redis 2.2.14
Rhoconnect 3.0.0

All above software are installed on servers by executing the following commands:

sudo yum install make patch gcc-c++ zlib-devel curl-devel pcre-devel readline-devel openssl-devel
sudo rpm -i rhoconnect-3.0.0.noarch.rpm

Tested bench application (benchapp)

Simulates creating multiple objects
100 concurrent users
1 device per user
20 object dataset
each object has 7 attributes

Prepare bench application for running on servers

Go to rhoconnect/bench directory and configure benchapp application:

sudo cp -R benchapp /opt/nginx/html
cd /opt/nginx/html/benchapp
sudo /opt/rhoconnect/bin/bundle install –without=test development
sudo mkdir public
sudo chown -R nginx:nginx /opt/nginx/html/benchapp

Edit ‘/opt/nginx/conf/conf.d/rhoconnect.conf’ file and point root entry to /opt/nginx/html/benchapp/public directory:
```
server {
listen      80;
root  /opt/nginx/html/benchapp/public;
passenger_enabled on;
}
```

Start redis and nginx servers:

sudo /etc/init.d/redis start
sudo /etc/init.d/nginx start

Benchmark results for large (2 cores) server

Request clientcreate   : min: 0.0058, max: 1.0360, avg: 0.1138, err: 0, verification err: 0
Request ack-cud        : min: 0.0258, max: 0.1998, avg: 0.0642, err: 0, verification err: 0
Request clientlogin    : min: 0.0056, max: 0.0958, avg: 0.0128, err: 0, verification err: 0
Request get-cud        : min: 0.0234, max: 0.1777, avg: 0.0476, err: 0, verification err: 0
Request create-object  : min: 0.0526, max: 0.3074, avg: 0.1242, err: 0, verification err: 0
Throughput(req/s)  : 8.40951725551376
Throughput(req/min): 504.571035330826

Benchmark results for extra large (4 cores) server

Request clientcreate   : min: 0.0080, max: 1.0194, avg: 0.0944, err: 0, verification err: 0
Request ack-cud        : min: 0.0298, max: 0.1381, avg: 0.0573, err: 0, verification err: 0
Request clientlogin    : min: 0.0091, max: 0.0591, avg: 0.0152, err: 0, verification err: 0
Request get-cud        : min: 0.0259, max: 0.1679, avg: 0.0443, err: 0, verification err: 0
Request create-object  : min: 0.0618, max: 0.2778, avg: 0.1112, err: 0, verification err: 0
Throughput(req/s)  : 8.19117682939103
Throughput(req/min): 491.470609763462

Benchmark results for high-cpu extra large (8 cores) server

Request clientcreate   : min: 0.0083, max: 1.0162, avg: 0.0741, err: 0, verification err: 0
Request ack-cud        : min: 0.0277, max: 0.1803, avg: 0.0525, err: 0, verification err: 0
Request clientlogin    : min: 0.0092, max: 0.0681, avg: 0.0147, err: 0, verification err: 0
Request get-cud        : min: 0.0259, max: 0.1729, avg: 0.0474, err: 0, verification err: 0
Request create-object  : min: 0.0599, max: 0.2850, avg: 0.1102, err: 0, verification err: 0
Throughput(req/s)  : 8.08355941878833
Throughput(req/min): 485.0135651273

All above results produced by bench script cud_script hosted by large (2 cores) EC2 server.