Palladio Build Infrastructure

Weiterleitung nach:

PCM Development/Build Infrastructure

Palladio Developer Wiki

Documentation restructuring

This page is still under construction. The following items should be addressed:

* Start with integration scenarios and describe each by example
* Describe the Palladio Build Infrastructure using Maven Tycho
* Describe the parameters provided by the target platforms
* Describe how to create a new build job on the SDQ jenkins build server
* Describe migration to GitHub and building GitHub projects on Jenkins
* Describe the central Palladio Update sites (for releases and for nightly builds)
* Best would be to have a systematic central entry page - best would be to have this one
* Have subpages for the detailed information outlined above

Integration Scenarios

PCM Addon

PCM addons extend the PCM core or extend another existing addon. In order to improve usability, all addons should be integrated in the Palladio build process and should be shipped on the Palladio update site. This means that the addon has to provide at least one feature that can be installed. The addon typically uses existing PCM features to provide its functionality.

You have to follow the steps described in Creating a new Addon or Support Project to integrate the addon in the build process.

Supporting Projects

Supporting projects do not depend on PCM features but are used by PCM. This means that they can be used in other projects as well. Even if such projects could be maintained in other contexts, we suggest to treat them like PCM addons to ensure compatibility with recent PCM releases.

You have to follow the steps described in Creating a new Addon or Support Project to integrate the project in the build process.

Code Improvements

Improvements fix bugs or introduce new features to existing PCM components. Integrating these changes does not require adjusting the build process but just contributing the code to the corresponding repository. Please refer to the description in Code Contributions.

PCM Product

Products are standalone versions of Eclipse with a pre-installed set of features. Users can use products easily by just extracting a ZIP file and running Eclipse. We integrate products in our build process in order to ensure up-to-date versions of the included features.

You have to follow the steps described in TODO: {{{1}}} to integrate the product in the build process.

Third Party Dependency

Palladio facilitates the Eclipse plugin infrastructure to manage dependencies. Unfortunately, not all required dependencies are available as Eclipse plugins. Therefore, these dependencies have to be wrapped. We favor wrapping these dependencies into Eclipse plugins rather than including them in the requiring plugin as JAR, which is also possible.

Please refer to the description in Palladio Build Infrastructure#Adding new Third Party Dependencies for some detailed instructions.

Build Infrastructure

Build infrastructures are made of code repositories, build management tools, and build servers. We are in the progress of migrating all of these components to more recent and commonly used tools. The old infrastructure consisted of SVN, Buckminster, and Jenkins template build jobs. The new infrastructure consists of GIT, Maven Tycho, and Jenkins pipeline scripts. We only describe the new infrastructure here because new projects must not use the old tools anymore.

Github

All Palladio source code is located in the PalladioSimulator Github organization. A rule of thumb is that there is a separate repository for every project that has an update site. To maintain clearness, we apply a naming scheme to all repositories and use Github topics to group repositories.

The naming schema is Palladio-Category-Name-Name2. Categories can be:

Category	Description
Core	Features that are essential for Palladio and can never be omitted.
Addon	Features that introduce new features to PCM but that do not fit in any other category.
Analyzer	Features that derive information about quality properties for given PCM models.
Simulation	Features that provide the simulation framework for analyzing PCM models.
QuAL	Features that record analysis results and provide analyses that derive quality metrics.
Editors	Features that provide editors for PCM models.
Bench	Projects that provide PCM features bundled read to use by users.
Supporting	Features that do not depend on PCM but are used by other Palladio features.
Build	Projects that support the build process including quality control of build results.

The topics we use on Github are:

Topic	Description
incubation	Experimental projects that are usually not yet shipped in Palladio releases.
third-party	Forked projects maintained or extended by Palladio developers.
build-system	Build-related projects.

For further settings of a new repository, please refer to our Palladio Github Repository Template.

Maven Tycho

Maven Tycho is an extension to Maven for building Eclipse plugins, features, products, and update sites. In order to use it, you only have to install Maven. The extension covers the usual build lifecycle including source generation, compilation, tests and packaging. We created extensions for Tycho that handle JavaDoc generation, target platform definition, and commony build descriptions.

The goal of using Maven Tycho is to create reproducible builds that do not depend on the execution environment but only on the build description. Therefore, we use explicit target platform definitions that contain fixed versions of dependencies. Additionally, we include a common base target platform for the execution environment in our parent POM. You can lookup the used target platform via the parent POM by looking for a property called org.palladiosimulator.maven.tychotprefresh.tplocation.0. This will, most probably, refer to our target platform artifact. This target platform will be merged with project-specific target platforms during the build. The base target platform contains all bundles of the Eclipse Modeling Edition bundle in our targeted Eclipse version (e.g. Oxygen). The merging is done by a build lifecycle extension we created.

Besides merging, the extension can refresh versions of features referenced in a target platform and can filter location entries by keywords. You have to add the attribute refresh="true" to the location tag in the target platform to refresh all versions of this entry. You can add a filter term to the location entry by adding the attribute filter="term". Predefined terms are release and nightly. The former is active during release builds, the latter is active during all other builds. You can add custom terms and add them to accordingly named properties to activate them. This process is covered in the readme of the extension.

Jenkins Pipeline Scripts

We use Jenkins as our build server that covers build and deployment of Palladio artefacts. Palladio builds always execute the same tasks for different input data. Therefore, we created reusable pipeline scripts that describe the build process in an abstract way. When defining a build job, we just fill in a few parameters to create a concrete build jobs.

The pipeline performs the following tasks

checkout source code
build the code by calling mvn clean verify
- includes running tests
- includes running checkstyle
- includes creating javadoc
deploy update site
archiving build artefacts
archiving quality metrics
- checkstyle
- junit results
- test coverage
mail notification about build result

The mandatory build parameters are:

gitUrl (https checkout url)
webserverDir (relative to the eclipse base folder on the webserver without leading and trailing slash)
updateSiteLocation (relativ path to the repository folder in the update site project's target folder without leading and trailing slash)

There are optional build flags that control the build process:

skipCodeQuality (default: false, disables collection of code quality metrics)
skipNotification (default: false, disables notifications of build results on the build mailing list)

The script can be modified on Github. Before every build job, the latest version from the master branch will be used.

Builds are triggered via Github triggers and time-based triggers.

Redirecting HTTP Proxy

We experienced severe reliability issues of the official Eclipse update sites during some periods over the year. This leads to failing builds. To prevent such failures, we use a HTTP proxy on our build server that redirects calls to Eclipse update sites to a mirror of these sites hosted at a university in the DFN network. We expect this to improve the reliability of our builds. Please note that the build definition still works as it is without using the proxy.

The proxy is defined in a new Global Maven settings.xml with the following content

<?xml version="1.0" encoding="UTF-8"?>
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0" 
          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
          xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd">
  <proxies>
    <proxy>
      <id>eclipseProxy</id>
      <active>true</active>
      <protocol>http</protocol>
      <host>localhost</host>
      <port>3128</port>
    </proxy>
  </proxies>
</settings>

We configure this proxy to be used for all Maven builds on our build server via the Global Tool Configuration. We use the global settings provider for maven to inject the mentioned settings.xml into our builds.

Contribution Activities

Creating a new Addon or Support Project

If you are doing a greenfield project, the simplest way is to adhere to the following steps in order to comply with the Palladio standards.

Generate a project stub using the Palladio Maven Archetype
Sort your Eclipse artifacts into the bundles and features folder
Define the target platform and the update site in the releng folder
- Please note that the build infrastructure already provides a target platform including the recent modeling tools dependencies of Eclipse as well as our branding and license plugins
- Ensure that you either only use release repositories of Palladio or include both release and nightly dependencies using the filter keyword (you can have a look at examples on how to do this)
- Refer to our licensing feature and our branding plugin in all your Eclipse features (see examples of existing projects)
Push everything into our PalladioSimulator organisation on Github
Create a new Jira project
Create a build job on our build server and use other build jobs as template
Add the build job to the nightly build job
External contributors:
- Contact us at palladio-dev mailing list and tell us about your addon
- Transfer ownership of your Github repository to our PalladioSimulator organisation
- Ask for creating a build job and a Jira project on our build server via the mailing list

Code Contributions

If you want to contribute to our existing projects, you can simply do this via the usual Github workflow. This means, you simply fork the project you want to contribute to. You work on the repository as you would usually do. Finally, you create a pull request. A Palladio developer will have a look at your contribution and decides if it should be included. If not, you will get some feedback on what should be improved to comply with our coding standards.

Adding new Third Party Dependencies

There are two ways of adding a third party dependency to Palladio. You can either configure a Maven to Eclipse conversion tool or wrap the dependency by yourself. We favor wrapping Maven plugins because this ensures that no modifications of the dependency took place (which might be important to fullfil licensing conditions) and it requires less code.

In order to wrap Maven dependencies, you can proceed like for contributing Code Contributions. You have to edit the pom.xml file in the ThirdParty library project. You can have a look at existing dependencies to identify how to add the dependency. It is crucial to add the license information in the POM file. TODO: Add link to compatible licenses.

TODO: Describe adding dependency to ThirdPartyWrapper