Reverbrain wiki

Site Tools


elliptics:serverside

Server-side processing engine

Definition

Server-side (former known as server-side scripting) is an code execution environment created in elliptics on demand. We use Cocaine for this. This allows us to dynamically create pool of cgroups-bound processes or set of LXC containers to execute externally loaded code, which is triggered by client command. One may consider this as a write trigger, but actually it is more than that - it is a special command which may have your data attached and with whole access to local storage as well as any other external elements.

For example you may want to connect to external MySQL servers and trigger special command which will read or write data into elliptics only when special record is valid in SQL database (not that it could not be implemented in elliptics only, but for the sake of example simplicity).

Worker types

Cocaine server-side pool can execute code in Python, Perl, Javascript (proof-of-concept) and binary compiled into shared library object (*.so). It is implemented through cocaine plugins (libcocaine-plugin-*) which are loaded by cocaine core (started in elliptics) on demand.

Applications

When you want your code (whether it is script in Python or compiled shared object) to be started in elliptics cloud you have to create an application, which our server-side engine Cocaine understands. Application is a simple noncompressed tarball which is uploaded into storage with associated configuration file which describes what it is and how to run your code.

For more details you may want to check our server-side tutorial

Initialization steps

Cocaine engine must be enabled and initialized in elliptics config first. Then you have to load your code into elliptics in a way cocaine understands. In current (0.9) version this is done by cocaine-deploy tool. When you start your application (actually when first trigger event is received by elliptics), pools of proper workers are created and code starts. You may want to read detailed tutorial for step-by-step setup.

Events

Serverside processing in elliptics is triggered by special command which operates with events. Event is basically a 'application-name@event-name' string with associated raw data embedded into elliptics packet. For every 'application-name' we start new cocaine engine (with its pools and so on).

Blocked events

Event can be blocked (if special flag is set) - in this case client blocks until event is fully processed (executed function returns) and error code as well as optional data is returned back to client from elliptics node. If event does not block, successful return means it was queued for execution.

Event pipeline

It is possible to create the whole pipeline where each node sends new events to another nodes and so on over some kind of signal-slot topology spread over the whole cluster for parallel realtime execution. Grape is a framework which greatly simplifies this task getting the whole work of network transport away from programmer, who only need to implement processing nodes and register them via signal-slot topology model.

elliptics/serverside.txt ยท Last modified: 2012/12/24 22:34 (external edit)