Question
So far I studied your project’s idea (Hierarchical Cluster Engine) and Doxygen descriptions, solution has a goal to allow to create distributed data storage. I can be wrong, Sphinx is used to process search along this storage.
What is final goal of HCE and PHP API implementation?
Answer
Yes, it is HCE supports distributed Sphinx search indexes internally.
But, this is only first of possible usage of HCE.
The DRCE (Distributed Remote Command Execution) functionality will be ready soon.
By this solution the HCE can to construct and manage freely structured network clusters as a distributed data and computations.
The main idea is to give the possibility to execute some artifact (binary executable, script on any program language, whatever…), get income source data and return resulted processed data with some reducing from down nodes to upper using any kind of standard internally supported or custom algorithms of sorting, elimination, grouping and so on…
In general HCE – is a network infrastructure support transport with data management (see main article) on target data nodes, reducing algorithms and of course it supports Sphinx search native way.
The main networking in the HCE based on ZMQ sockets solution (externally and internally for inter-thread data transfer).
So, all kind of request/response network interactions are based on ZMQ client library for any language.
The main goal of PHP design is to create OOP binding implementation correspondent to existing structural code for PHP language.
Also, we planned to create a client side API bindings for many popular languages like Python, Java, Ruby, Perl and so on…
Please, check the implementation of some utilities, like import to key-value DB storage: http://hierarchical-cluster-engine.com/docs/dox…
Also, the searcher and manager utility implementation that are supports most of operations that can be done by HCE core engine.
They are uses ready to use but have a structural API.
May be this helps to understand the subject area more…
Question
Got your idea. Do you have some vectors to advertise HCE? onference, forums? Habrahabr? š
May be some competitive exhibitions?
So, if we know this, we’ll be able to build a pull of tasks with priorities. So after fixed time there will be some ready box-product, for example. Or, probably, I’m trying to think about things you already thought.
Answer
The HCE project now is a pilot implementation for open source community.
But, we have internal customer inside the IOIX corp., so we are not a team of investors, but have possibility to do some research investigations in measures of available budget and applied needs of current main project called the SNATZ (http://snatz.com) that using the ASM v2.0 for internal full-text engine tasks.
Now the HCE project site and components like Debian Linux packages and tarballs – are under finish of release v1.0 and official open date is 2014-01-01.
Of course i’ll publish some information articles news on most of well known sites like IT news in application area before the HCE site will be opened.
Because the team is small and applied tasks eats many time i cant to estimate real plans on box-product now. But, after some community discussion i’ll start to plan some more precise view of set of features that can be formed as a box product and distributed as complete solution. But now it is a Lego puzzle that can be used by some professionals as a tool that has specific cover and possible very good implementation for their project infrastructure.
Off coarse there are several competitors on marked now.
All of them are build by huge companies with great possibilities and most of them uses Java or completely implemented on Java platform.
Now, we cant to have some comparable position, so it is very wrong idea to compare HCE with the Storm or Hadoop…
But, power of huge projects is some imperfection because they need resources to manage and support as well as complex closed internal behavior and logic.
More light and flexible tool possible can to give advantages of functionality that can be freely extended, modified, tuned and even – redesigned for special needs with relatively small resources. (I hope…)