State of the open source project management

Ideas
#1

Hi,

i wanted to start a thread here, because i was thinking about the state of the open source development process of the platform lately. Especially the solution to my problem on this
topic got me thinking about.

So what i thought about here is how the development in the open works from the perspective of an outsider. Since it has been already about 6 month since the first open source release (which i found pretty amazing btw.), i thought we (as a community) should have a conversation about the current state of how we “work” together to bring the platform forward in one form or the other.

For me at least it was especially interesting to see the reasons for open souring the platform in this very
honest and open article.

So, perhaps we can use this thread for everyone to either give their two cents on what could be improved and brainstorm and discuss on ideas.

Here are my opinions on what could be improved:
Generally i looked at an article about successful github development

public issue tracker

This is a point that is quite important in my opinion. I sometimes look at the commit history on github and thinking to myself that it would be quite beneficial to not only look at the result, but also look at the underlying discussion of a commit that it relates to. Since i’m not into the details of the different classes that are part of the commit its quite interesting to look at the commits, but probably not the most efficient way to get an idea what this commit should do or in which context it has to be placed.

Additionally i think it makes it much easier to track which bugs / improvements will make it in which release.

When i look at other open source projects that are developed “in the open” like Spring Data JPA or Rails or EntityFramework, a well structured issue tracker can make a huge difference. I found the term “develop in the open” that a quite good description of the mindset behind this kind of development which i first heard about when Microsoft (Scott Hanselman) annouced .NET as OSS.

What we have here in this forum is in some sense the same idea, but (for good reasons) much more unstructured. I sometimes found it a bit problematic that when a potential bug is contributed, there is no transparent path between the submit and the resolve of an bug (although you lately added comments when implemented in a particular version, which is quite a good starting point).

As i imagine, this is not only one of the most important points, but probably one of the most exhausting things for your to implement as well. Since it requires major changes in the way you work internally and what information are now publicly available and would be afterwards is kind of hard i could imagine.

contribution information

When i look at it, the project has in comparison to other project a fairly low pull requests to commits ratio.
I think this could potentially be improved by writing down more information about how to contribute,
what types of tests you expect, potential things like coding styles etc.

Additionally a lot of open source projects even encourage participation through something like labels on issues
that indicate good starting points for beginners (in Rails it’s called good-first-patch).

public CI system for testing

having a public CI system for running the tests gives another level of confidence in the project. Additionally its required
when starting to working “in the open”. Since something like travis is for free for open source projects,
it shouldn’t be a big deal, right?

public roadmap

Last thing that comes to my mind for now is the issue of a missing roadmap (as asked here).
Although spotty information are valuable as well, an updated roadmap on a very high level would be very good
to get a good grasp on what is planned in the future, what features will be added and also what features will not be part of the platform
anymore.

So these are my ideas. I hope not only Haulmont employees could add their opinions here (although i would appreciate that as well), but also users from the outside like me, since my opinions are probably not representitive for the outside world :slight_smile:

Bye,
Mario

#2

Hi Mario,

Thank you for this important (though complicated) question.

The platform was an internal proprietary tool at Haulmont for years, so we are just learning to be open-source now, and your suggestions are very valuable. We still think more about features and releases than about community and project management - but we are going to balance these things better.

First, let me explain my personal attitude to this matter (maybe someone from the team will comment too). I think that the main point for the platform to be open source is to gain trust from the users - trust that it will not disappear or get too expensive. I don’t believe much in code contributions because the majority of our users are probably people of the type who just wants to get things done. But I understand that the level of confidence directly depends on the size of the community and on the level of openness, which is defined also by the management things you’ve mentioned.

So let me consider your ideas one by one.

public issue tracker

We now have an instance of YouTrack which holds the issues for all projects at Haulmont, including the platform. It is integrated with other internal tools like GitLab, TeamCity and Upsource. We are happy with this configuration and wouldn’t like to change it radically, at least in the near future.

There are the following problems with giving public access to the platform YouTrack:

  • We don’t want to pay an extra license fee for external users.

  • We now often write ticket descriptions and comments in Russian - only titles are 100% in English. It saves us time and gives better understanding of the topic.

  • Our IT team is paranoid about the security of our internal services.

I think we could give an anonymous read-only access to YouTrack (if IT decides that it is safe enough), so everyone will be able to see all content, including target milestone, fix versions, etc. Next task would be to make all bug reporters and developers write English only.

So you are right, this is the most painful question, but we’ll try to move on.

contribution information, public CI system for testing

The main problem here is that the main test suite is outside of the free core platform - it’s in a separate application (RefApp) which embraces all premium add-ons, even the old Workflow, as it is used in some internal projects. So in our internal CI environment, we first build and test the platform, then all add-ons and RefApp. Moreover, there are UI tests on Selenium and DB update tests that run nightly.

Perhaps we will just add more information on how to use the tests contained in the core platform, but the rest of testing will be still on our side.

public roadmap

We’ll certainly do this, as a first thing after releasing 6.3 - probably next week. And of course, we’ll be happy to discuss it with all who are interested.

Thanks again!

Would be great to see comments on this topic from other users as well :slight_smile:

#3

Hi Konstantin,

I don’t believe much in code contributions because the majority of our users are probably people of the type who just wants to get things done

This is an interesting statement. I think you might be right when it comes to users who wants to use the platform for creating their software, “just” for getting stuff done. This might be the case for internal software and alike (but i might be wrong on that). But i’m really not sure, if this are the majority of the (future-) user base. Especially i think when it comes to users who will use the software as a basis for their own software. In this case, there is probably a much bigger possibility that these users are in the position to contribute not only to their own software layer, but the underlying infrastructure as well.

As an example of this you can look at the docker project. The majority of the user base of this piece of software is probably, as you said, not interested in pulling the golang sources and get their hands dirty in the details of the linux kernel mechanisms. Because they act on a total another level of abstraction. They “just” want to get their container running and use docker only as the plumbing that has to be done.
Nevertheless, on the other hand it does not at all mean, that there are already around 1500 individual contributors where a major proportion of this contributions come from “OEM” companies like Google or Red Hat that build their own technologies on top of it and therefore are willing to act as additional contributors. But enough of somewhat misleading comparisons :slight_smile:

But the main point you did lay out yourself: There is a correlation between openness and trust. Obviously putting the sources out on github was the main trust builder and everything additional is exactly that: an minor addition (at least from a users perspective)…

The points regarding the issue tracker and test cases are somewhat related to the above mentioned. I totally understand your points and really can put myself in your shows there. I think it boils down to the major question: Are you generally interested in not only users but in developers as well*. This interest can come because you think that this will help *you (as a company) or because it will help the project. Of course these process are quite painful and also, as a matter of fact, it will decrease the optimization the internal workflows you’ve established, because in the end of this process, the platform could be more something like an “external dependency”, or at least more external that it used to be before.

I think it’s a very good sign that there were words like “at least in the near future” etc, because it is more important to have a direction everyone agrees on that concrete steps like “getting the world access to your private issue tracker”.

But let’s hear some other opinions on that…

Bye,
Mario

#4

Hi, I’ve landed here from github (after writing this issue ) and I have more or less the same concerns Mario had 7 months ago… and the fact nobody else wrote on this topic is not reassuring!

I’m primarily a developer, and to me the true “openness” relies more on the governance of a project, than a repo or two with an OSS license attached to it!

I never found a framework or tool in my entire career that satisfied my needs 100%, without some minor or major tweaking of its internals. So it’s important that the entire codebase is well documented, and not only from a “user” perspective… In this regard I see CUBA lacks a low-level reference, like class hierarchies, javadocs etc. and low-level details on the internal architecture/modules (or maybe I missed some part of your docs?).

If I find something missing, or a feature that needs improvements to work the way I need, I want to be able to actively contribute it, in a collaborative way, and if everything goes ok to open a PR with that code.

Do you plan to accept external contributions in the short term? How about someone that opens a PR on GH?
Are you considering writing a guide for core contributors, with all the internal details needed to grasp the entire codebase?

To me this is a major show-stopper before I start adopting it for important projects, or spread the word in public talks or meetups for example…

Thanks,
Paolo

#5

Hi Paolo,

We are definitely planning to accept external contributions. We have already moved our YARG library and its development process to GitHub. At the moment we are preparing our local CUBA repository to migrate to GitHub, so entire development will be performed directly on GH. I believe that we will complete it within the next 2 weeks, so all the Work in Progress activity, feature branches and PRs will be public and available on GitHub.

Currently, there are no contribution guidelines and public documentation for internals of the platform, but we really want to go in this direction.

We appreciate you for refreshing this topic and happy to hear that our thoughts of moving to GH resonates with the ideas of our community members. Thank you.

2 Likes
#6

Thanks Yuriy for your clarifications. These are definitely the kind of news I was hoping! :wink:
And yes, I agree that moving to GH (despite its shortcomings, if compared to more mature PM tools) is surely the right thing to do for the success of an OSS project.

Regarding the “basics” of contribution guidelines, CLAs et al, you could take inspiration from the Angular or Aurelia repos (GitHub - angular/angular: The modern web developer’s platform). They are the most “disciplined” repos I’ve ever found, with issue templates, clear coding guidelines, commit messages format, etc.

I hope to be able to contribute ASAP (maybe a first-class kotlin support), but I need to complete my first project first, to see the full potential of CUBA in action…

Paolo