Notes from Daitan Group on Symphony Integrations Development

Daitan Group provides software product development services to leading global technology companies including Symphony and contributed significantly to the Symphony Integrations project, which is one of the top 3 most active projects hosted by the Symphony Software Foundation.

In the spirit of sharing experiences with the community around open source best practices, we want to describe the development process that the team has crafted on top of the Open Developer Platform (ODP) delivered by the Foundation, walking through each step from the developer’s perspective.

Source Code Management (SCM)

The first step is to get (or update) the project source code from GitHub; the configuration of the SCM is crucial for the correct development of the project, as it delivers:

1. A place to store all of the project’s code assets, organize it in different GitHub repositories and have a landing documentation page on each of them (README.md)
2. History and conflict management across all team member activity.
3. The way to configure and enforce the branching model (We use feature-driven development, see Github flow to know more about what the best fits your team).
4. IP compliance checking, using the Foundation’s cla-bot.
5. Full Admin access (to preserve project self-governance) to project leads, so that they can configure anything in GitHub without the need of opening issues (and wait) for Foundation Staff.
6. Documentation generation, using Github Pages in combination with static website engines; you can check the generated documentation.

Building the Code

The first activity on top of a checked-out codebase is to run the build process, which must be ensured to work in any circumstance and environment (in the functional boundaries of the project). This is why it is crucial to define and maintain a working build process.

For the Symphony Integrations project we used Java technologies and adopted Apache Maven as our build tool, which allowed us to reuse the Foundation’s Maven Parent Pom. This gave us tons of plugin configurations for the following build features:

1. Orchestrates Java and JavaScript builds to allow deploying/releasing frontend and backend as a single component
2. Delivers build-time integrations with code validation tools
3. Enforces binary artifact versioning (including nightly builds, aka snapshot artifacts)
4. Configures binary artifact deployment on Maven Central (and Sonatype OSS for snapshots)
5. Provides easy-to-use release capabilities

Normally, the build process runs on developers workstations, which can be misconfigured or have configuration issues which leads to a build failure. To prevent this from happening, the Foundation provides documentation and infrastructure to easily set up a continuous integration build process. We chose to use Travis CI, which delivers:

1. Containerized and isolated builds
2. Advanced build configurations (ie invoking mvn and npm on the same build)
3. Authentication and authorization inherited by GitHub (no need to sign up)

Validating the Code

Given the context of the project - integration for THE messaging platform for the financial world, we must ensure that the quality, security and legal compliance of the software is continuously scanned and validated. For this purpose, the Foundation ODP provides several code validation tools, depending on the language and use case you’re trying to solve.

The main tool used for security and legal validation is WhiteSource, as it supports the majority of the use cases we are interested in and delivers integration with all the languages we use (Java and NodeJS), specifically:

1. Check libraries for problematic/undefined licenses.
2. Generates legal reports.
3. Check libraries for security vulnerabilities.
4. Check libraries for bugs.
5. Check libraries for outdated versions.
6. Check libraries for release activity.

We enforce the security aspect by adopting CoverityScan, which adds static code analysis capabilities at build time:

1. Scans code for security vulnerabilities.
2. Scans code for bad practices.
3. Scans code for bugs.

Last but not least, we enforce software quality - in terms of code activity and test coverage - by using CodeCov.

Running the Code

The ultimate step is to continuously deploy the code on a running service, which is normally referred to as continuous delivery. The Foundation provides a containerized infrastructure that can be easily integrated with Travis CI to ensure that every successful build also triggers the deployment of the binaries.

The Value of Open Source Foundations

Working in open source environments has multiple benefits. For example, consider the differences between developing in a proprietary source environment versus going open source. In a proprietary environment we often find that systems code becomes more tightly coupled codebases and controlled by a few individuals leading to interdependencies that can slow release cycles, impacting efficiency and (potentially) a business’ competitiveness. In open source environments there is a higher adoption of best practices and community collaboration, leading to more modular designs which enables greater agility and independence to the development team.

Second, the symbiotic structure provided by the the Symphony Software Foundation ensures neutral oversight for a Project Lifecycle and guidance toward key milestones that enforce quality assurance, validation and governance of all contributions. Within this collaborative framework, each contributor maintains full control over individual code assets while adding incremental value to the total project and community stakeholders.

Conclusions

It’s been a great experience to work with the Foundation and their ODP infrastructure, as it has optimized our workflow efficiency by adopting cutting-edge yet industry-approved services, tools and technologies and offloaded several tedious infrastructure setup tasks accelerated our performance, especially during the initial stages of the project. We ensured quality by adopting automated testing from the very first day, with multiple tools for code validation and security,

Finally, Daitan Group has always advocated open source and community-driven initiatives, so we plan to continue our efforts with the Foundation, hoping we provide valuable contributions for others.

Checkout the Symphony Integrations documentation and drop a comment below if you have any questions.