ChiliProject is not maintained anymore. Please be advised that there will be no more updates.

We do not recommend that you setup new ChiliProject instances and we urge all existing users to migrate their data to a maintained system, e.g. Redmine. We will provide a migration script later. In the meantime, you can use the instructions by Christian Daehn.

How to synchronize two instances of ChiliProject/Redmine

Added by Andreas Schuh at 2012-03-27 02:25 pm

Hi ChiliProject Folks,

I posted this question also on the Redmine forum, but was wondering if any of you who do not check the Redmine forum frequently would have a solution to this problem. In fact, ChiliProject is yet my personal favorite which we are considering to use.

The problem is that due to security considerations and restrictions on our public web server, we are thinking about running two instances of ChiliProject/Redmine, one publicly accessible for our released software and one running on our internal server. All projects would have an entry in the internal ChiliProject/Redmine database. Everything that is posted to the public ChiliProject/Redmine instance should automatically also appear in the internal ChiliProject/Redmine instance. Due to firewall restrictions, we would need to have a periodic process running which pulls all changes and integrates them into the internal database. Moreover, all updates to entries in the internal database which have been initially created through the public instance would need to be synchronized back.

Scenario 1: One of our lab members creates a new issue using the internal instance. This issue only appears in the internal database and shall not be integrated in the public database.
Scenario 2: A user of our software creates a new issue using the public instance. This issue should be integrated in the database of our internal instance such that it is accessible via the internal web site. Otherwise, we would need to check two project pages for updates.
Scenario 3: An issue created by a user which has been integrated into the internal database is updated by us, for example, setting status to Closed. This update must be reintegrated into the public database such that the user can see it and be notified.

Any ideas on how this could be achieved? A solution on the database level which is actually rather independent of ChiliProject/Redmine would be desired.

Thank you,
Andreas


Replies (6)

RE: How to synchronize two instances of ChiliProject/Redmine - Added by Felix Schäfer at 2012-03-28 07:09 am

Andreas Schuh wrote:

Any ideas on how this could be achieved? A solution on the database level which is actually rather independent of ChiliProject/Redmine would be desired.

That's complicated at best because you're then dependent on the exact schema of the DB, the data in the DB is tightly coupled together, and I guess even things like auto increment columns will be hard to get done in an async manner (instance A claims "the next" ID, instance B claims "the next" ID too before they are synced, what now?). Long story short, I don't know of any solution to have ChiliProject or Redmine instances periodically synchronized.

Shameless plug: we (finn) might be able to help, feel free to contact us for more information.

RE: How to synchronize two instances of ChiliProject/Redmine - Added by Jan Vlnas at 2012-04-03 09:05 pm

I am currently working on a similar problem for our faculty.

What I have in mind is a remote tracking of bugs, similar to the feature in Launchpad or Bugzilla. You just add a link to the remote bug and the bugtracker fetches more details.
This could be implemented as a plugin, possibly with some bot periodically fetching and updating bugs statuses using REST API.

RE: How to synchronize two instances of ChiliProject/Redmine - Added by Andreas Schuh at 2012-04-04 03:48 am

Yes, I see now that the solution must be at the application level (see also reply by Gert in the Redmine forum).

Querying changes from the public database A to add them also to the private database B, as well as to synchronize existing rows which correspond to the same record. Here of course one needs to establish such correspondence between the database rows with differing primary key (e.g., issue ID). Therefore, I was thinking about adding a column to the tables of interest in the private database B, which would indicate the corresponding rows in the tables in database A. Then, either a plugin or a process which makes use of the REST API would be responsible to keep corresponding rows in sync, respectively, to add new rows in database A also to database B. Basically, this sounds doable, but I fear that as usual the crux is in the detail and requires more efforts than we can afford for this matter. One problem to deal with would probably be conflicting updates in both databases, even though I do not expect too frequent updates on the public end, i.e., database A. So I was just hoping there would be already something out there that would reduce our efforts and, not to forget, reduce the risk of leading towards a dead end...

Therefore, I would in first place focus on what we actually need to have synchronized. In first place, we are interested in the issues view related tables. So it would be a start to have only these "synchronized". Again, I would not require that the issue IDs on both ends are identical for the very same issue, only that from database B I have a reference to the corresponding issue in database A.

That the schema of both databases would be tightly coupled is not an issue here as the idea is to run the same version of ChiliProject/Redmine at both ends with the same database schema, but simply differing database content.

RE: How to synchronize two instances of ChiliProject/Redmine - Added by Jan Vlnas at 2012-04-04 10:30 am

Andreas Schuh wrote:

[…] Here of course one needs to establish such correspondence between the database rows with differing primary key (e.g., issue ID). Therefore, I was thinking about adding a column to the tables of interest in the private database B, which would indicate the corresponding rows in the tables in database A.

You can keep this on application level too, ChiliProject/Redmine already provides custom fields functionality (such as Remote issue URL used by ChiliProject's bugtracker) – this information is also exposed through REST API.

The risk of conflicting updates depends on how much data you want to synchronize. Syncing issue's status (or other fields) is easy, you just pick the most recent issue and update its status on the other instance of CP. REST API also provides details on which fields were changed, see e.g. https://www.chiliproject.org/issues/826.xml?include=journals
Changes done through API are also logged and users are notified, which is an advantage of a highlevel approach.
If you also want to sync comments, the plugin/bot would probably need to keep information on which comments to which issues were already added – but synchronization is still quite easy given you just add stuff.

RE: How to synchronize two instances of ChiliProject/Redmine - Added by Andreas Schuh at 2012-04-04 03:07 pm

The most promising solution thus far which indeed is already implemented seems to be rubyrep as pointed out by Holger.

RE: How to synchronize two instances of ChiliProject/Redmine - Added by Alexey Java at 2012-06-16 01:00 am

Task Adapter (http://www.taskadapter.com) can transfer tasks between Redmine and Chiliproject instances. If you need more control over what's transferred, you have to create a custom transformer using https://github.com/taskadapter/redmine-java-api

(1-6/6)