Reverbrain wiki

Site Tools


elliptics:major-version-updates

Table of Contents

Major version updates

v2.26

  • Multiple backends per elliptics node support.
    Backend is eblob/filesystem entity living in own directory with own cache, thread pools, status and so on. This allows to have single elliptics node per large machine with multiple backends added per disk. Previously we suggested to add new elliptics node per disk. How to apply this feature you can read in backends section of the configuration guide.
  • Split route table update to address/ids commands.
    This greatly reduces network traffic needed to keep route table up to date. Implemented massively parallel asynchronous connection/route table update.
  • Added multiple new automatic build-time tests: dc/merge recovery, backend control, route table update.
  • New structured Blackhole logger.
  • Monitoring system supports all kinds of statistics now, some of them were not available previously (per-backend config and vfs stats). Allow monitoring to group multiple request types into single request.
  • C++ callback logic rewritten to reduce older C code bloat as well as greatly simplify codebase. Added parallel and quorum lookup types.
  • Introduce client-side/server-side errors.
    Let's consider write operation. For example, client perform write to 3 nodes and one of them is unavailable. In previous versions of Elliptics in this situation client will have 2 answers instead of 3. Now client gets 3 answers but one of it (for unavailable node) is local (client-side) and answer has a special bit that allows to understand whether answer from server or from client.

v.2.25

  • Monitoring, extended statistics, custom providers (backend, cache, queues, per-command).
    Detailed description of Monitoring features you can find in project documentation.
  • SLRU cache, improved parallelization.
  • Index capped collections.
  • Doxygen docs.
    All documentation as possible is written in code, just the most common topics are in external sources.
  • Collection of various autotests (c++, cocaine, python, API, server-side, timeouts).
    Each new feature and much of older ones now have build-time tests. This leads to longer build time but enlarge the confidence of build consistency.
  • C++/Python API improvements. Also improved API of Cocaine “elliptics”-storage plugin.
  • Recovery improvements (performance, state saving, recovery of keys listed in special file, dump recovery states, continue from checkpoints).
    Continue from checkpoints is a very useful feature for big storages where recovery procedure can lasts for some days. If such a long recovery process is failed by any reason you can continue it from the checkpoint and save much time.
    If your storage is big some keys can stay inconsistent for a long time due to eventual consistency of elliptics storage. If you need strong consistency of some keys you can use recovery of keys listed in special file.
  • Range iterators.
    When you need to iterate through the storage, you can use ranges of keys instead of iterating by all keys. If you understand why you need specific keys range, you will get large gain in time.
  • Read weights adjustment/selection algorithm.
    This is a great improvement of cluster performance because you will get data mostly quickly from the least loaded node.
  • New config format.
    Now configuration file is in JSON. It is more comfortable for autogeneration/parsing purposes.
  • Parallel connect, route table update optimizations.
    This is a great feature that dramatically scales down time for cluster startup. It is especially effective in high latency networks and in case of large amount of unavailable nodes.
  • Uneven ranges rebalancing.
    This feature is provided with an utility. This utility redistributes keys in ranges between nodes in hash ring to make these ranges even if they are not.
    Uneven ranges almost surely occur in case of a few amount of keys at several nodes.
  • Trace Id.
    This is a logger feature that simplifies parsing of logs.
  • Bugfixes.

v.2.24 and earlier

  • Elliptics as it is.
elliptics/major-version-updates.txt · Last modified: 2014/08/19 22:15 by bayonet