Warning Older Docs! - You are viewing documentation for a previous released version of RhoMobile Suite.

Synchronization with Rhodes

As we’ve shown in the Rhom section, adding synchronized data via RhoConnect to your Rhodes application is as simple as generating a model and enabling a :sync flag. This triggers the internal Rhodes sync system called the SyncEngine to synchronize data for the model and transparently handle bi-directional updates between the Rhodes application and the RhoConnect server.

Since version 4.0 SyncEngine API is available through ‘rhoconnect-client’ extension, which is bundled as part of RhoMobileSuite in ‘rhoconnect-client’ gem. All new-generated Rhodes applications will include rhoconnect-client extension by default. For existing applications that use SyncEngine API you should add ‘rhoconnect-client’ to the extensions section of build.yml.

This section covers in detail how the SyncEngine works in Rhodes and how you can use its flexible APIs to build data-rich native applications.

Sync Workflow
Sync Authentication
Notifications
Binary Data and Blob Sync
Filtering Datasets with Search
SyncEngine API
SyncEngine AJAX API
Backround synchronization on iOS

Sync Workflow

The SyncEngine interacts with RhoConnect over http(s) using JSON as a data exchange format. With the exception of bulk sync, pages of synchronized data, or “sync pages” as we will refer to them here, are sent as JSON from RhoConnect to the SyncEngine.

Below is a simplified diagram of the SyncEngine workflow:

This workflow consists of the following steps:

SyncEngine sends authentication request to RhoConnect via SyncEngine.login. RhoConnect calls Application.authenticate with supplied credentials and returns true or false.
If this is a new client (i.e. fresh install or reset), the SyncEngine will initialize with RhoConnect:
- It requests a new unique id (client id) from RhoConnect. This id will be referenced throughout the sync process.
- It will register platform information with RhoConnect. If this is a push-enabled application application, the SyncEngine will send additional information like device push pin.
SyncEngine requests sync pages from RhoConnect, one model(or Rhom model) at a time. The order the models are synchronized is determined by the model’s :sync_priority, or determined automatically by the SyncEngine.

Sync Authentication

When you generate a Rhodes application, you’ll notice there is an included directory called app/Settings. This contains a default settings_controller.rb and some views to manage authentication with RhoConnect.

`login`

In settings_controller.rb#do_login, the SyncEngine.login method is called:

SyncEngine.login(
  @params['login'], 
  @params['password'], 
  url_for(:action => :login_callback) 
)

Here login is called with the login and password provided by the login.erb form. A :login_callback action is declared to handle the asynchronous result of the SyncEngine.login request.

`login_callback`

When SyncEngine.login completes, the callback declared is executed and receives parameters including success or failure and error messages (if any).

def login_callback
  error_code = @params['error_code'].to_i
  if error_code == 0
    # run sync if we were successful
    WebView.navigate Rho::RhoConfig.options_path
    SyncEngine.dosync
  else
    if error_code == Rho::RhoError::ERR_CUSTOMSYNCSERVER
      @msg = @params['error_message']
    end

    if not @msg or @msg.length == 0   
      @msg = Rho::RhoError.new(error_code).message
    end

    WebView.navigate( 
      url_for(:action => :login, :query => {:msg => @msg}) 
    )
  end  
end

This sample checks the login error_code, if it is 0, perform a full sync and render the settings page. Otherwise, it sets up an error message and re-displays the login page with an error.

`application.rb#on_sync_user_changed`

If the SyncEngine already knows about a logged-in user and a new user logs in, then the on_sync_user_changed hook is called (if it exists) before the login_callback. This is useful, for example, if you want to re-initialize personalized settings for a new user.

require 'rho/rhoapplication'

class AppApplication < Rho::RhoApplication
  def initialize
    super
  end

  def on_sync_user_changed
    super
    MyCoolApp.reset_user_preferences!
  end 
end

If on_sync_user_changed, data for all sync-enabled models will be removed. To remove data for all local models as well:

def on_sync_user_changed
  super
  Rhom::Rhom.database_local_reset
end

Other auth-related methods are described in the SyncEngine API section.

Notifications

The SyncEngine system uses notifications to provide information about the sync process to a Rhodes application. Notifications can be setup once for the duration of runtime or each time a sync is triggered. One a sync is processing for a model, notifications are called with parameters containing sync process state. Your application can use this information to display different wait pages, progress bars, etc.

To set a notification for a model, you can use the following method:

SyncEngine.set_notification(
  Account.get_source_id,
  url_for(:action => :sync_notify),
  "sync_complete=true"
)

Which is the same as:

Account.set_notification(
  url_for(:action => :sync_notify),
  "sync_complete=true"
)

In this example, once the sync process for the Account model is complete, the view will be directed to the sync_notify action (with params ‘sync_complete=true’) if user is on the same page.

In these examples, after the sync is complete the notifications are removed.

You can also set a notification for all models:

SyncEngine.set_notification(
  -1, url_for(:action => :sync_notify), 
  "sync_complete=true"
)

This notification will not be removed automatically.

Notification Parameters

When the notification is called, it will receive a variable called @params, just like a normal Rhodes controller action.

Common Parameters

These parameters are included in all notifications.

@params["source_id"] - The id of the current model that is synchronizing.
@params["source_name"] - Name of the model (i.e. “Product”)
@params["sync_type"] - Type of sync used for this model: “incremental” or “bulk”
@params["status"] - Status of the current sync process: “in_progress”, “error”, “ok”, “complete”, “schema-changed”

“in_progress” - incremental sync

@params["total_count"] - Total number of records that exist for this RhoConnect source.
@params["processed_count"] - Number of records included in the sync page.
@params["cumulative_count"] - Number of records the SyncEngine has processed so far for this source.

“in_progress” - bulk sync

@params["bulk_status"] - The state of the bulk sync process:

“start”: when bulk sync start and when specific partition is start syncing

“download”: when client start downloading database from server

“change_db”: when client start applying new database

“blobs”: when client start downloading remote blob files

“ok”: when sync of partition finished without error

“complete”: when bulk sync finished for all partitions without errors

@params["partition"] - Current bulk sync partition.

“error”

@params["error_code"] - HTTP response code of the RhoConnect server error: 401, 500, 404, etc.
@params["error_message"] - Response body (if any)
@params["server_errors"] - Hash of Type objects of RhoConnect adapter error (if exists): “login-error”, “query-error”, “create-error”, “update-error”, “delete-error”, “logoff-error”

For “login-error”, “query-error”, “logoff-error”: Type object is hash contains ‘message’ from server: @params[“server_errors”][“query-error”][‘message’]

For “create-error”, “update-error”, “delete-error”: Type object is hash each containing an “object” as a key (that failed to create) and a corresponding “message” and “attributes”: @params[“server_errors”][“create-error”][object][‘message’], @params[“server_errors”][“create-error”][object][‘attributes’]

“create-error” has to be handled in sync callback. Otherwise sync will stop on this model. To fix create errors you should call Model.on_sync_create_error or SyncEngine.on_sync_create_error

“ok”

@params["total_count"] - Total number of records that exist for this RhoConnect source.
@params["processed_count"] - Number of records included in the last sync page.
@params["cumulative_count"] - Number of records the SyncEngine has processed so far for this source.

“complete”

This status returns only when the SyncEngine process is complete.

“schema-changed”

This status returns for bulk-sync models that use FixedSchema when the schema has changed in the RhoConnect server.

In this scenario the sync callback should notify the user with a wait screen and start the bulk sync process.

Server error processing on client

create-error

has to be handled in sync callback. Otherwise sync will stop on this model. To fix create errors you should call Model.on_sync_create_error or SyncEngine.on_sync_create_error:

SyncEngine.on_sync_create_error( src_name, objects, action )
Model.on_sync_create_error( objects, action )

* objects - One or more error objects
* action - May be :delete or :recreate. :delete just remove object from client, :recreate will push this object to server again at next sync.

update-error

If not handled, local modifications, which were failing on server, will never sync to server again. So sync will work fine, but nobody will know about these changes.

SyncEngine.on_sync_update_error( src_name, objects, action, rollback_objects = nil )
Model.on_sync_update_error( objects, action, rollback_objects = nil)

* objects - One or more error objects
* action - May be :retry or :rollback. :retry will push update object operation to server again at next sync, :rollback will write rollback_objects to client database.
* rollback_objects - contains objects attributes before failed update and sends by server. should be specified for :rollback action.

delete-error

If not handled, local modifications, which were failing on server, will never sync to server again. So sync will work fine, but nobody will know about these changes.

SyncEngine.on_sync_delete_error( src_name, objects, action )
Model.on_sync_delete_error( objects, action )

* objects - One or more error objects
* action - May be :retry - will push delete object operation to server again at next sync.

For example: :::ruby SyncEngine.on_sync_create_error( @params[‘source_name’], @params[‘server_errors’][‘create-error’], :delete)

SyncEngine.on_sync_update_error( @params['source_name'], 
    @params['server_errors']['update-error'], :retry)
SyncEngine.on_sync_update_error( @params['source_name'], 
    @params['server_errors']['update-error'], :rollback, @params['server_errors']['update-rollback'] )

SyncEngine.on_sync_delete_error( @params['source_name'], 
    @params['server_errors']['delete-error'], :retry)

unknown-client error

Unknown client error return by server after resetting server database, removing particular client id from database or any other cases when server cannot find client id(sync server unique id of device). Note that login session may still exist on server, so in this case client does not have to login again, just create new client id. Processing of this error contain 2 steps:

When unknown client error is come from server, client should call database_client_reset and start new sync, to register new client id:

  rho_error = Rho::RhoError.new(err_code)

  if err_code == Rho::RhoError::ERR_CUSTOMSYNCSERVER
    @msg = @params['error_message']
  end

  @msg = rho_error.message unless @msg and @msg.length > 0   

  if rho_error.unknown_client?(@params['error_message'])
    Rhom::Rhom.database_client_reset
    SyncEngine.dosync
  end

If login session also deleted or expired on the server, then customer has to login again:

  rho_error = Rho::RhoError.new(err_code)

  if err_code == Rho::RhoError::ERR_CUSTOMSYNCSERVER
    @msg = @params['error_message']
  end

  @msg = rho_error.message unless @msg and @msg.length > 0   

  if err_code == Rho::RhoError::ERR_UNATHORIZED
    WebView.navigate( 
      url_for(
        :action => :login, 
        :query => { :msg => "Server credentials expired!" } 
      )
    )   
  end

Notification Example

Here is a simple example of a sync notification method that uses some of the parameters described above:

def sync_notify
  status = @params['status'] ? @params['status'] : ""
  bulk_sync? = @params['sync_type'] == 'bulk'

  if status == "in_progress"    
    # do nothing

  elsif status == "complete" or status == "ok"
    WebView.navigate Rho::RhoConfig.start_path

  elsif status == "error"

    if @params['server_errors'] && 
       @params['server_errors']['create-error']
        SyncEngine.on_sync_create_error( @params['source_name'], 
            @params['server_errors']['create-error'], :delete)
    end

    err_code = @params['error_code'].to_i
    rho_error = Rho::RhoError.new(err_code)

    if err_code == Rho::RhoError::ERR_CUSTOMSYNCSERVER
      @msg = @params['error_message']
    end

    @msg = rho_error.message unless @msg and @msg.length > 0   

    if rho_error.unknown_client?(@params['error_message'])
      Rhom::Rhom.database_client_reset
      SyncEngine.dosync

    elsif err_code == Rho::RhoError::ERR_UNATHORIZED
      WebView.navigate( 
        url_for(
          :action => :login, 
          :query => { :msg => "Server credentials expired!" } 
        )
      )                
    else
      WebView.navigate( 
        url_for(
          :action => :err_sync, 
          :query => { :msg => @msg } 
        )
      )
    end    
  end
end

If the view was updated using AJAX calls, this mechanism may not work correctly as the view location will not change from one AJAX call to another. Therefore, you might need to specify the :controller option in WebView.navigate.

Sync Object Notifications

The SyncEngine can also send a notification when a specific object on the current page has been modified. This is useful if you have frequently-changing data like feeds or timelines in your application and want them to update without the user taking any action.

To use object notifications, first set the notification callback in application.rb#initialize:

class AppApplication < Rho::RhoApplication
   def initialize
    super

    SyncEngine.set_objectnotify_url(
      url_for(
        :controller => "Product",
        :action => :sync_object_notify
      )
    )
   end
end

Next, in your controller action that displays the object(s), add the object notification by passing in a record or collection of records:

class ProductController < Rho::RhoController

  # GET /Product
  def index
    @products = Product.find(:all)

    add_objectnotify(@products)
    render
  end

  # ...

  def sync_object_notify
    #... do something with notification data ...

    # refresh the current page
    WebView.refresh
    # or call System.execute_js to call JavaScript function which will update list
  end
end

Object Notification Parameters

The object notification callback receives three arrays of hashes: “deleted”, “updated” and “created”. Each hash contains values for the keys “object” and “source_id” so you can display which records were changed.

Binary Data and Blob Sync

Synchronizing images or binary objects between RhoConnect and the SyncEngine is declared by having a ‘blob attribute’ on the Rhom model. Please see the blob sync section for more information.

Filtering Datasets with Search

If you have a large dataset in your backend service, you don’t have to synchronize everything with the SyncEngine. Instead you can filter the synchronized dataset using the SyncEngine’s search function.

Like everything else with the SyncEngine, search requires a defined callback which is executed when the search results are retrieved from RhoConnect.

Using Search

First, call search from your controller action:

def search
  page = @params['page'] || 0
  page_size = @params['page_size'] || 10
  Contact.search(
    :from => 'search',
    :search_params => { 
      :FirstName => @params['FirstName'], 
      :LastName => @params['LastName'], 
      :Company => @params['Company'] 
    },
    :offset => page * page_size,
    :max_results => page_size,
    :callback => url_for(:action => :search_callback),
    :callback_param => ""
  )
  render :action => :search_wait
end

Your callback might look like:

def search_callback
  status = @params["status"] 
  if (status and status == "ok")
    WebView.navigate( 
      url_for( 
        :action => :show_page, 
        :query => @params['search_params']
      ) 
    )
  else
    render :action => :search_error
  end
end

Typically you want to forward the original search query @params['search_params'] to your view that displays the results so you can perform the same query locally.

Next, the resulting action :show_page will be called. Here we demonstrate using Rhom’s advanced find query syntax since we are filtering a very large dataset:

def show_page
  @contacts = Contact.find(
    :all,
    :conditions => { 
      {
        :func => 'LOWER', 
        :name => 'FirstName', 
        :op => 'LIKE'
      } => @params[:FirstName], 
      {
        :func => 'LOWER', 
        :name=>'LastName', 
        :op=>'LIKE'
      } => @params[:LastName],
      {
        :func=>'LOWER', 
        :name=>'Company', 
        :op=>'LIKE'
      } => @params[:Company],
    }, 
    :op => 'OR', 
    :select => ['FirstName','LastName', 'Company'],
    :per_page => page_size, 
    :offset => page * page_size 
  )    
  render :action => :show_page
end

If you want to stop or cancel the search, return “stop” in your callback:

def search_callback    
  if(status and status == 'ok')
    WebView.navigate( url_for :action => :show_page )
  else
    'stop'
  end
end

Finally, you will need to implement the search method in your source adapter. See the RhoConnect search method for more details.

SyncEngine API

Below is the full list of methods available on the SyncEngine:

`login(login, password, callback)`

Authenticates the user with RhoConnect. The callback will be executed when it is finished. See the authentication section for details.

SyncEngine.login(
  @params['login'], 
  @params['password'], 
  url_for(:action => :login_callback) 
)

`logout`

Logout the user from the RhoConnect server. This removes the local user session. See the authentication section for details.

SyncEngine.logout

`logged_in`

Returns 1 if the SyncEngine currently has a user session, 0 if not.

if SyncEngine::logged_in == 1
  render :action => :index
else
  render :action => :login
end

`dosync(show_sync_status = true, query_params = "", sync_only_sources_with_local_changes = false )`

Start the SyncEngine process and display an optional status popup (defaults to true). query_params will pass to sync server. sync_only_sources_with_local_changes indicates that only sources that have local changes will be synced.

SyncEngine.dosync(false)
  #=> no status popups are displayed

SyncEngine.dosync(false, "param1=12&param2=abc")
  #=> no status popups are displayed and parameters will pass to sync server

SyncEngine.dosync(false, "", true)
  #=> no status popups are displayed and synchronization will be performed only for sources with local changes.

`dosync_source(source_id_or_name, show_sync_status = true, query_params = "")`

Star the SyncEngine process for a given source id or source name and display an optional status popup (defaults to true). query_params will pass to sync server

SyncEngine.dosync_source(Product.get_source_id.to_i, false) #sync by source id
SyncEngine.dosync_source(Product.get_source_name, false) #sync by source name

`lock_sync_mutex`

Blocking call to wait for SyncEngine lock (useful for performing batch operations).

SyncEngine.lock_sync_mutex
#... perform blocking tasks...
SyncEngine.unlock_sync_mutex

`unlock_sync_mutex`

Release the acquired SyncEngine lock (make sure you do this if you call lock_sync_mutex!).

`stop_sync`

Stops any sync operations currently in progress.

SyncEngine.stop_sync
  #=> no callback is called

`set_notification(source_id, callback_url, params = nil)`

See the sync notification section.

`set_notification(-1, callback_url, params = nil)`

Set notification callback for all models. This callback is not removed after the sync process completes. See the sync notification section.

`clear_notification(source_id)`

Clears the sync notification for a given source id.

SyncEngine.clear_notification(Product.get_source_id)

`on_sync_create_error( src_name, objects, action )`

“create-error” has to be handled in sync callback. Otherwise sync will stop on this model. To fix create errors you should call Model.on_sync_create_error or SyncEngine.on_sync_create_error.

SyncEngine.on_sync_create_error( @params['source_name'], 
    @params['server_errors']['create-error'], :delete)

objects - One or more error objects
action - May be :delete or :recreate. :delete just remove object from client, :recreate will push this object to server again at next sync.

`on_sync_update_error( src_name, objects, action, rollback_objects = nil )`

SyncEngine.on_sync_update_error( @params['source_name'], 
    @params['server_errors']['update-error'], :retry)
SyncEngine.on_sync_update_error( @params['source_name'], 
    @params['server_errors']['update-error'], :rollback, @params['server_errors']['update-rollback'] )

objects - One or more error objects
action - May be :retry or :rollback. :retry will push update object operation to server again at next sync, :rollback will write rollback_objects to client database.
rollback_objects - contains objects attributes before failed update and sends by server. should be specified for :rollback action.

`on_sync_delete_error( src_name, objects, action )`

SyncEngine.on_sync_delete_error( @params['source_name'], 
    @params['server_errors']['delete-error'], :retry)

objects - One or more error objects
action - May be :retry - will push delete object operation to server again at next sync.

`set_pollinterval(interval)`

Update the SyncEngine poll interval. Setting this to 0 will disable polling-based sync. However, you may still use push-based-sync.

SyncEngine.set_pollinterval(20)'
  #=> now polls every 20 seconds

`set_syncserver(server_url)`

Sets the RhoConnect server address and stores it in rhoconfig.txt.

SyncEngine.set_syncserver("http://myapp.com/application")
  #=> don't forget the '/application' path

`set_objectnotify_url(url)`

See the sync notification section.

`set_pagesize(size)`

Set the sync page size for the SyncEngine. Default size is 2000. See the SyncEngine workflow for how this is used.

SyncEngine.set_pagesize(5000)

`get_pagesize`

Get the current sync page size for the SyncEngine. See the SyncEngine workflow for how this is used.

SyncEngine.get_pagesize
  #=> 2000

SyncEngine.set_pagesize(5000)
SyncEngine.get_pagesize
  #=> 5000

`enable_status_popup(false)`

Enable or disable show status popup. True by default for Blackberry, false for other platforms.

SyncEngine.enable_status_popup(true)

`set_ssl_verify_peer(true)`

Enable or disable verification of RhoConnect ssl certificates, true by default.

# using a self-signed cert
SyncEngine.set_ssl_verify_peer(false)

`get_user_name`

Returns current username of the SyncEngine session if logged_in is true, otherwise returns the last logged in username.

SyncEngine.get_user_name
  #=> "testuser"

`search(*args)`

Call search on the RhoConnect application with given parameters. See the search section for more details.

# :from             Sets the RhoConnect path that records 
#                   will be fetched with (optional).
#                   Default is 'search'.
#
# :search_params    Hash containing key/value search items.
#
# :offset           Starting record to be returned.
#
# :max_results      Max number of records to be returned.
#
# :callback         Callback to be executed after search
#                   is completed.
#
# :callback_param   (optional) Parameters passed to callback. 
#
# :progress_step    (optional) Define how often search callback 
#                   will be executed with 'in_progress' state. 
# :sync_changes     (optional)  - true or false(default). Define should client changes send to server before search.

Contact.search(
  :from => 'search',
  :search_params => { 
    :FirstName => @params['FirstName'], 
    :LastName => @params['LastName'], 
    :Company => @params['Company'] 
  },
  :offset => page * page_size,
  :max_results => page_size,
  :callback => url_for(:action => :search_callback),
  :callback_param => "",
  :sync_changes => false
)

`search(*args) (multiple sources)`

Call search on the RhoConnect application with multiple source names. This is useful if your search spans across multiple models.

For example:

SyncEngine.search(
  :source_names => ['Product', 'Customer'],
  :from => 'search',
  :search_params => { 
    :FirstName => @params['FirstName'], 
    :LastName => @params['LastName'], 
    :Company => @params['Company'] 
  },
  :offset => page * page_size,
  :max_results => page_size,
  :callback => url_for(:action => :search_callback),
  :callback_param => "",
  :sync_changes => false
)

Parameters are the same as for ModelName.search with an additional parameter:

:source_names - Sends a list of source adapter names to RhoConnect to search across.

SyncEngine AJAX API

Sync engine AJAX API has been implemented to provide access to low-level control on synchronization process right from plain HTML/javascript UI pages.

It isn’t intended to be used in every application. It requires deep knowledge of SyncEngine functionality and operations. It may broke your application if used improperly so use it with care please.

Backround synchronization on iOS

On iOS, if application is put to background, it will be suspended. To allow application finish sync after application goes to background, you can use ‘finish_sync_in_background’ parameter in rhoconfig.txt. When this parameter is set to ‘1’, if sync is active in the time of background transition ( e.g. started from app_deactivate handler ), application will not be suspended until sync is finished.