PerOpteryx

Aus SDQ-Wiki
Palladio Addon (Liste)
Name PerOpteryx
Contacts Anne Koziolek (http://are.ipd.kit.edu/people/anne koziolek/)
State Stable
Is Stand-alone Analysis? No
Extends Analyses SimuCom, PCM2LQN, PCM-based Reliability Prediction, PCM Cost Solver
Extends Metamodels Annotations
Code Location https://github.com/PalladioSimulator/Palladio-Addons-PerOpteryx
Update Site https://sdqweb.ipd.kit.edu/eclipse/palladio/addons/peropteryx/nightly/
JIRA https://jira.palladio-simulator.com/projects/PEROPTERYX

Short Summary

PerOpteryx is an optimization framwork to improve component-based software architectures based on model-based quality prediction techniques.

Documentation

Currently, PerOpteryx is realised as a set of Eclipse plugins to automatically improve Palladio Component Model instances for performance, reliability, and cost properties. Conceptually, it is independent of the considered quality attributes, software architecture metamodelling language, and degrees of freedom; and can be applied to improve any software architecture models using any quantitative quality prediction technique.

PerOpteryx includes a Quality of service Modeling Language (QML, Frolund 1998) model to express quality requirements to steer the optimization. See QML Quickstart for details and references.

Bat.jpg        Peropt-logo.jpg

PerOpteryx feature exploration extension is described on a separate page.

Background

Bat.jpg

PerOpteryx stands for "Performance Optimizer" and is named after a bat, because it shall explore the architectural design space like a bat explores physical space with sonar.

The tool takes a complete PCM instance as input and produces new PCM instances by manipulating certain parameters of the model based on a genetic algorithm. As a new and distinctive feature, it uses domain-specific knowledge (e.g. performance tactics) to improve the generic evolutionary optimisation. PerOpteryx analyses the quality attributes (e.g. performance and costs) of the new instances and tries to find the Pareto front of optimal trade-off solutions. It helps software architects to make informed trade-off decisions between different, conflicting quality attributes.

PerOpteryx works on top of the PCM Bench and uses PCM2LQN, LQNS, and PCMMarkov for performance and reliability prediction. It also features a simple cost calculator that can be used to model any type of costs (e.g. development costs, procurement costs, operating costs, or total cost of ownership). Analysis tools for other quality attribute can be readily added as Eclipse plugins using Eclipse's extension point mechanism. PerOpteryx is implemented based on the Opt4J framework and uses the efficient, multi-objective NSGA-II genetic algorithm internally.

The tool is currently being developed at the Architecture-driven Requirements Engineering Group of the Karlsruhe Institute of Technology in Germany. For details and questions about PerOpteryx, please contact Anne Koziolek.

Documentation

Publications

Publications discussing different aspects of the approach and case studies (see http://sdq.ipd.kit.edu/people/anne_koziolek/publications/ for full bibliographic details and bibtex entries).


Core PerOpteryx Publications

The best starting point is the first PerOpteryx publication:

Other Core PerOpteryx Publications

  • ICPE'12: An industrial case study of performance and cost design space exploration (PDF), corresponding article in the "ABB Review" magazine, page 35 et seqq.
  • QoSA'11: Peropteryx: Automated application of tactics in multi-objective software architecture optimization (ICSA 2021 10 Years Most Influential Paper Award). (PDF), details on case studies can be found at PerOpteryx/Tactics Case Study.
  • CBSE'11: Towards a generic quality optimisation framework for component-based system models. (PDF)
    • Described the generic design space formulation used to optimize software architectures modelled in any architecture modelling language for any quantitatively predictable quality attribute.
    • Metamodel-independent formulation of the degrees of freedom
  • QUASOSS'09: Optimising multiple quality criteria of service-oriented software architectures (PDF)
    • Early workshop paper including reliability prediction into PerOpteryx
  • FESCA'09: Automatic, model-based software performance improvement for component-based software designs (PDF)
    • Early workshop paper proposing a hill-climbing approach, only considers performance, superseded by above papers.
  • WCOP'08: Performance-oriented Design Space Exploration (PDF)
    • First presentation of the idea at a workshop, only considers performance, superseded by above papers.

Extension to not-quantified quality attributes

  • Max Scheerer, Axel Busch, and Anne Koziolek. Automatic Evaluation of Complex Design Decisions in Component-based Software Architectures. (DOI)
  • Axel Busch and Anne Koziolek. Considering Not-quantified Quality Attributes in an Automated Design Space Exploration. (DOI)

Thesis

Most detail is provided by the dissertation thesis on PerOpteryx, (PDF).

Extension to Requirements Engineering

  • TwinPeaks 2012: Architecture-driven quality requirements prioritization. (PDF)
  • REFSQ'12: Research preview: Prioritizing quality requirements based on software architecture evaluation feedback. (PDF)

Related Publications

Other publications related to PerOpteryx or extending it:

  • Michele Ciavotta, Danilo Ardagna, and Anne Koziolek. Palladio optimization suite: Qos optimization for component-based cloud applications. (DOI)
  • ICPE'11 and JSS'14: Detection and solution of software performance antipatterns in palladio architectural models ICPE (DOI) JSS DOI, ICPE best paper award. Joint work with Vittorio Cortellessa and Catia Trubiani from University of L'Aquila, Italy.
  • Workshops and Symposia at MODELS 2010: Focussing multi-objective software architecture optimization using quality of service bounds (revised and extended version of QUASOSS paper below) (PDF)
  • QUASOSS'10: Using Quality of Service Bounds for Effective Multi-objective Software Architecture Optimization (PDF)
    • Initial work to extend PerOpteryx to consider bounds for quality requirements to focus the search on interesting regions
  • QoSA'10: A Hybrid Approach for Multi-Attribute QoS Optimisation in Component Based Software Systems. Joint Work with Danilo Ardagna and Raffaela Mirandola from Politecnico Milano, Italy. (PDF)
  • FASE'10: A process to effectively identify ”guilty” performance antipatterns. Joint work with Vittorio Cortellessa and Catia Trubiani from University of L'Aquila, Italy. (PDF)

Tool Documentation

The tool contains some help content. Click the question mark in the Eclipse Run Configuation (shown below).

PerOpteryx run config.PNG

Degrees of Freedom

The current implementation supports these degrees of freedom

  • Component selection for components that offer at least the same interfaces than the current one and require exactly the same interfaces (deletion of AssemblyConnectors is not implemented yet).
  • Component allocation (no check that servers have to offer all required resource types yet implemented)
  • Continuous Processing Rate change: the processing rate of any resource can be continuously varied in a given interval. Note that the MTTF is adjusted linearly together with the rate: Higher rates lead to higher MTTF, too, because processors are more expensive and more carefully produced (this could as well be modelled the other way around, but is not configurable yet).
  • Scheduling Policies for processing resources can be adjusted
  • Capacity of passive resources can be adjusted.
  • Replication of servers: Replicas of servers can be created for load balancing (not in the core PerOpteryx release, but available on an svn branch), see ICPE'12 publication (PDF).

More degrees of freedom for component-based architectures are discussed in our WOSP paper and in Anne's thesis (see above).

These generic degrees of freedom (GDoF) are instantiated automatically for a concrete system at hand as shown in this example. Each GDoF has a matching rule that detects instances of itself:

PerOpteryx-DoFInstances.png

Each degree of freedom instance (DoFI) has a set of possible options. In this tiny example, we have two components to use at place C in the architecture, namely component C1 (grey) and component C2 (green). Let us further assume we have three components to use at place D, namely D1, D2, and D3. Additionally, we can allocate each of the components to other servers (let us assume we have one more server S2 not shown here). Finally, we could change the server speed of the servers S1 and S2. Thus, we get

  • Component selection DoFI at C with options C1 and C2
  • Component selection DoFI at D with options D1, D2, and D3
  • Component allocation DoFI at C with options S1 and S2
  • Component allocation DoFI at D with options S1 and S2
  • Server speed DoFI at S1 with options [1GHz, 3GHz]
  • Server speed DoFI at S2 with options [1GHz, 3GHz]

The choice of values are the genotype of a candidate. The next figure shows an excerpt of the DoFI.

PerOpteryx-EvaluateCandidate.png

Additionally, the figure shows how to evaluate the quality properties for a candidate, i.e. for a vector of chosen values from the design space spanned by the DoFI. A model transformation applies the chosen values (the genotype) to the initial system model (lower arrow). The transformation rules are defined generically by the GDoF. Then, the generated model (the phenotype) can be evaluated using standard techniques, resulting in the predicted quality properties. This translation from genotype to phenotype is made transparent for the optimisation algorithm, which just sees that a set of choices (a genotype) can be evaluated to quality properties (uper arrow).

The images are taken from the WOSP talks, which can be downloaded as pdf or animated powerpoint slides above.

Note that a DoFI does not only represent the update of a single variable in the model. For example, when replacing a component in the PCM, one has to update the place in the architecture (called AssemblyContext) to bind the new component and additionally has to update the connectors so that they connect the right interfaces. In principle, a DoFI can be any model transformation. The restriction is that the model transformation must not change the model too much so that other DoFI become invalid. This is discussed in more detail in my thesis, which is (hopefully) available soon.

The figure below shows another example for the instantiation of DoFI from GDoF and a given system model. The manual input is the system model. Then, with the generic degree of freedom (GDoF) rules, the DoFI are automatically instantiated. Additionally, candidates canbe automatically evaluated (see picture above).

PerOpteryx-OutlineDesignSpaceSection.png

Cost Model

PCM models can be annotated with costs in a cost model file. The approach is simple: Repository components and resources (servers, i.e. processing resources and linking resources) can be annotated with costs. If the components or resources are used in a system, these costs can be added up. One can consider fixed costs (i.e. procurement costs) and/or operating costs. A discount factor can be used if both fixed costs and operating costs shall be used to calculate the net present value.

The EMF cost model directly contains the costs caldulations for components and resources. The de.uka.ipd.sdq.dsexplore.analysis.cost bundle contains the caldulation of the overall costs, i.e. determines which components and resources are actually used in the system (i.e. have a respective Assembly Context and Allocation Context) and adds up their costs.

Repository components can be annotated with "ComponentCost", so that these costs are added to the overall costs if the component is used in a system.

For resources, one can specify

  • fixed costs, which are added to the overall costs if the resource is used
  • variable costs, for which a function can be defined how the costs depend on the processing rate or linking resource property (TODO: check which ones).

The resulting types of costs are Component Cost, VariableProcessingResourceCost, VariableLinkingResourceCost, FixedProcessingResourceCost, and FixedLinkingResourceCost. For the VariableProcessingResourceCost, it is possible to specify a function for the cost based on the processing rate of that resource, e.g. 0.7665*((procRate.VALUE/2)^6.2539)+145 . The keyword procRate.VALUE is then replaced by the current processing rate of a resource.

Metric descriptions as used in other places in Palladio are used so far (this is also because the cost model is older than the metric descriptions).

Details for published and submitted papers

Download / Install / Use

For the latest features and improvements, use the PCM 4.0 release candidate of the PCM (or install the PCM Nightly). It should work with any common operating system. If you have any problems, please do not hesitate to contact us. For working on the PerOpteryx code and compiling it, e.g. when extending PerOpteryx as part of a student thesis, please also see PerOpteryx#Development below.

General requirements

PCM 4.0

Please install PCM 4.0 as described there and make sure to select the "Palladio DSE (PerOpteryx)" feature. If you have problems, try to select all available features from the PCM update site.

Nightly build

  • Please install PCM nightly as described there and select the "Palladio DSE (PerOpteryx)" feature. If you have problems, try to select all available features from the PCM update site.

Version 1.0.1 (not supported any more)

The update site is http://sdqweb.ipd.kit.edu/eclipse/palladio/addons/peropteryx/releases/core_3.3.1/1.0.1/. This version is based on the service release PCM 3.3.1. Enter both update sites in the "Install new Software" dialogue and check "PerOpteryx" and "Palladio Component Model", then install.

Version 1.0.1 on a Mac (not supported any more)

Thanks to Waldemar Flat for this information:

Für die Installation von Eclipse + PCM + PerOpteryx braucht man folgende Quellen:

Für die R Installation habe ich die Version 3.1.0 genommen (wurde von PCM verlangt). (Auch bei PCM 3.3.1):

Die folgenden Dateien habe ich wie folgt erweitert:

~/.profile:

# R installation
export R_HOME=/Library/Frameworks/R.framework/Versions/Current/Resources/
export PATH=$R_HOME/bin:$PATH

Eclipse.app/Contents/MacOS/eclipse.ini:

-Djava.library.path=/Library/Frameworks/R.framework/Versions/Current/Resources/library/rJava/jri

Starten unter MacOSX 10.9.3: Zum Starten von Eclipse muss die Datei „Eclipse.app/Contents/MacOS/eclipse“ verwendet werden. Bei einem Doppelklick auf die Eclipse.app-Datei wurde R (beim mir) nicht gefunden.

Older Versions (not supported any more)

The PCM 3.3 version and PerOpteryx 1.0 have some issues and are not recommended, but can still be used.

Use PerOpteryx

  • Start Eclipse and import a PCM project. For example, take the simple optimisation example from https://github.com/PalladioSimulator/Palladio-Example-Models/tree/master/SimpleHeuristicsExample_Example
  • Open the design decision file "simpleexample.designdecision" (the optimization currently only works if the design decision model has been loaded before in the editor).
  • Select "Run" -> "Run Configurations" from the main menu.
  • Click "PCM Design Space Exploration" to configure your PerOpteryx run.
  • While "PCM Design Space Exploration" is marked and you do not have any configuration active for editing, you can click the help button in the lower left orner to access the PerOptery help (see figure below).

PerOpteryx-HowTo-help.jpg]

First, you need to configure the model files. Enter the files names in the "model files" tab as usual. The only new thing for PerOpteryx is the QMLCriteria Definition, which allows you to define which criteria (mean response time, reliability, costs) you want to optimise with your system. Enter the qmldeclaration that comes with the example model as shown below. Alternatively, check this page. For details how to create a QML declaration file for your own model, refer to Qais Noorshams diploma thesis.

PerOpteryx-HowTo-model-files.jpg

In the second tab, you can configure the options for PerOpteryx. The screenshot below shows the most common setting: Choose a number of iterations that the search should run and the size of the population in each iteration. Choose "Evolutionary Search" in the drop down box below to optimise using genetic algorithms (the other two are for validation purposes).

The only other important setting in this tab are the "design decision options" at the bottom. You can either provide your own designdecision file (= degrees of freedom to use in the search) here or let PerOpteryx determine one.

  • Let PerOpteryx determine the designdecisions: Check the box "Only determine design decisions, do not optimise" to let PerOpteryx determine the designdecisions and let you review them before optimising. You need to provide a file name with full path here so that PerOpteryx knows where to write the designdecisions to. Open the file and check it, before proceeding to the third case in this list.
  • Let PerOpteryx determine the designdecisions and optimise them right away: Uncheck both checkboxes
  • Optimise your system with predefined designdecisions that are to be loaded from the given file. Check "Only optimise, using pre-existing design decisions" to make PerOpteryx optimise the system with the designdecisions loaded from the given file.

In the middle of this tab, you can make some specialised settings:

  • In the "Starting Population" field, you can provide a csv file with a staring population for PerOpteryx. If you do not provide any, the start population is randomly generated (the first candidate is always the system as you modelled it). A .csv file is expected. It has to have the same format as the optimisation results file (thus, run the optimisation first if you are not sure how they look like). You can, however, omit the objective values in the predefined candidates.

If you specify predefined candidates and set the maximum number of iterations to 0, then the tool will evaluate all candidates specified in the file, regardless of the specified "number of individuals per iteration" and of the specified search algorithm.

  • The other two possibilities to enter a file are 1) an option to provide objective values for a cache so that rerunning an optimisation can run faster (usually not needed) and 2) an option to continue an interrupted run, in that case not only the last population of the interrupted run needs to be provided as the start population (see above), but also the archive of the internal genetic algorithm NSGA-II needs to be provided. The archive is one of the result files of PerOpteryx.

PerOpteryx-HowTo-DSE-options.jpg

Finally, you can configure the quality analyses as usually.

In the performance analysis tab you can select SimuCom, LQNS or the LQN simulator. The figure below shows an example configuration for LQNS.

PerOpteryx-HowTo-performance-analysis.jpg

In the reliability analysis tab you can configure the Markov solver as usual. PerOpteryx-HowTo-reliability-analysis.jpg|400px]]

To analyse costs, you need a cost model for your system. See e.g. the example model shown in the figure below. Basically, you can assign costs to components in the repository (component costs) which will be added to the system cost if the component is used and thus allows to model costs for different component alternatives. Second, you can assign costs to servers. You can either assign fixed costs for a server, or Variable Costs that are adjusted depending on the processing rate of the server. The latter is useful if the processing rate of your servers is a designdecision (i.e., a degree of freedom) in your system. In that case you should use Variable Costs so that a faster resource indeed leads to higher costs. A cost model derived from data from Intel is 0.7665*((procRate.VALUE)^6.2539)+145 where procRate.VALUE is a keyword for PerOpteryx which is replaced by the current processing rate of the resource for a candidate. The processing rate in this case is assumed to be specified in GHz.

Check out the example cost model in the example (file ending .cost) to learn more.

PerOpteryx-HowTo-costs-analysis.jpg

The other tabs can be ignored. Their options should be unchecked in most cases.

Then, hit Run to start the optimisation! Note that to test it, it might be useful to configure a few iterations (e.g. 2) and LQNS, so that it finishes quickly.

PerOpteryx will output the results as csv files and designdecision files in a subfolder PerOpteryx_results of your current project (see figure below).

PerOpteryx-HowTo-where-results.jpg

For each iteration, three output files are created. Thus, even while the optimisation is still running, you can check preliminary results. The first number in the file name (before the underscore) is the iteration). The most important files are the archive*: They contain the results the optimisation found so far, i.e. the Pareto-optimal candidates of that iteration. The allCandidates* files contain all candidates evaluated up to that iteration. The population* files contain the population of the genetic algorithm in that iteration.

The csv files can be used again as input to the tool as described above. They look as follows:

PerOpteryx-HowTo-results-cvs.jpg

Each row contains a candidate architecture. The figure aboe shows the results in the archive (i.e., the Pareto-optimal candidates) after 10 iterations. The first columns contain the quality values for the candidates. Here, it is first POFOD, then costs, then mean response time. Then, the next columns are the design decisions of the candidate, i.e. which choices where made for the degrees of freedom for this candidate. In this example, we first have the Processing Rates of the three servers, then the allocation of the three components (the value in the field says to which server the component in the headline of the column is allocated to), and the next columns contain the choice of components, if any alternatives were given in the system. Here, two alternative components could be used in the AssemblyContext "Assembly_BookingSystem", there were two available options "BookingSystem" and "QuickBooking". The final columns give the utilisation of the servers.

You can draw a diagram in Excel to study the tradeoffs. Sort the results first if you have infinity values, because Excel cannot plot these. Then you can create a result plot like below.

PerOpteryx-HowTo-results-plot.jpg

Troubleshooting

If you copy components to create alternative components for the AssembledComponentDegreeofFreedom, make sure to manually edit the copied component's id and the ids of all model elements contained in it. EMF assumes unique ids of all model elements.

Screenshots

Example Models

Below you can find several PCM model instances, which you can use for playing around with Peropteryx.

Minimal Example

This is a minimal PCM instance with all annotations to perform a successful run of Peropteryx. It consists of three components and three servers.

If you want to use the predefined launc configurations, make sure that you installen the LQN solver tools (see PCM2LQN).

UML Visualization of the Minimal Example PCM instance         Layered Queueing Network generated from PCM instance

Business Reporting System

This is a larger PCM instance with 8 servers and 12 components. It models a Business Reporting System where clients can generate and download business reports from data stored in a database. The model has been used in this paper as a case study and is loosely based on a realistic system.

UML Visualization of the Business Reporting Server PCM instance         Layered Queueing Network generated from PCM instance

The LQN model has been created with the PCM2LQN tool (which is included in the latest PCM release). It contains multiple dummy processors and tasks, which were added to make the transformation from PCM more straightforward.

A comparison of the evolutionary optimisation with a random search for this case study is available on a separate page: PerOpteryx/BRSRandomSearch.

Quickstart

To get started with the optimisation, you need a valid QML model that defines your requirements on the PCM instance. This page shows a QML model quickstart.

Related Projects

Foundations

PerOpteryx applies a multiobjective genetic algorithm to the problem. Currently, NSGA2 as implemented in the Opt4J framework has been chosen.

Concerning multiobjective optimisation, the following work is useful as background information

  • Blum & Roli, Metaheuristics in combinatorial optimization: Overview and conceptual comparison (DOI), they discuss single-objective metaheuristics.
  • Ehrgott, Multicriteria Optimisation (Google Books link), he discusses multiobjective problems in general and presents algorithmic solutions.
  • Coello Coello, A Comprehensive Survey of Evolutionary-Based Multiobjective Optimization Techniques (DBLP Entry Google Scholar search)
  • Sean Luke, 2009, Essentials of Metaheuristics (available at http://cs.gmu.edu/~sean/book/metaheuristics/)
  • Harman et al., Search Based Software Engineering: A Comprehensive Analysis and Review of Trends Techniques and Applications, (PDF)

Development

Development Setup

PerOpteryx code is now available on Github:

Additionally, install the PerOpteryx nightly build as described above, to get all additional plugins PerOpteryx depends on.

The following plugins are not up to date, please do not import in your workspace or if you import them, keep them closed. In general, if you experience complile errors in plugins that you do not work on yourself, try closing the problematic plugins (you may not need them) and ask your advisor at your next meeting about it.

  • BayesNetworkOperator

The *.dsexplore.csvanalyzer project and all *taskmodel* related plugins are not needed when running PerOpteryx, so you can also close it.

If you want to use the *.dsexplore.csvanalyzer project to compare different PerOpteryx run results, you also need to import org.opt4j from https://svnserver.informatik.kit.edu/i43/svn/code/ThirdPartyWrapper/trunk into your workspace (the csvanalyzer is not a plugin and thus cannot resolve the dependency on its own).

Finally, before you update your Eclipse setup (PCM plugins, PerOpteryx plugins) using the update sites, please make a backup copy of your Eclipse directory. This way, if there is a problem in the nightly build of the PCM or PerOpteryx, you will be able to fall back to your previous working version and you will not be blocked.

For debugging PerOpteryx, we suggest to remove the console output limitation of your development Eclipse (Right click in console -> Preferences) so that you can see all debug output in the Eclipse console. However, for experiments with PerOpteryx, make sure to set the console output to a small value (e.g. 1000 characters) to save memory.

Getting Started

The core of PerOpteryx is the de.uka.ipd.sdq.dsexplore plugin. It has three main parts:

  • PerOpteryx uses Opt4J and implements the interfaces defined by Opt4J (Figure 4 in the Opt4J technical report). These can be found in all packages containing opt4j in their name.
    • PerOpteryx uses a special type of Genotype for handling degrees of freedom on software architecture models explicitly (instead of using an integer genome or similar). This is the DesignDecisionGenotype which is handles by DSECreator, DSEDecoder, etc. (all implementing Opt4J interfaces).
  • PerOpteryx is executed using the Palladio Workflow Engine. Classes related to launching PerOpteryx are located in the .launch package.
  • The different quality analysis are factored out, each in its own plugin. The package de.uka.ipd.sdq.dsexplore.analysis defined the interface for all these quality analysis plugins. Examples for these plugins are de.uka.ipd.sdq.dsexplore.analysis.cost, de.uka.ipd.sdq.dsexplore.analysis.simucom, and de.uka.ipd.sdq.dsexplore.analysis.lqn.

The actual degrees of freedom are defined in a metamodel in the de.uka.ipd.sdq.pcm.designdecision plugin. If you would like to add new types of degrees of freedom, you can either update the metamodel in this plugin or use the so-called Generic Degree of Freedom capability of PerOpteryx, which allows you to define degrees of freedom using OCL. This functionality is not yet available in the trunk of PerOpteryx, but in the branch at https://svnserver.informatik.kit.edu/i43/svn/code/Palladio/Addons/PerOpteryx/branches/GDOF_OCL_CodeAdaptation.

Team

Anne Koziolek (Martens) Ralf Reussner Heiko Koziolek Steffen Becker
Anne Koziolek (Martens) Ralf Reussner Heiko Koziolek Steffen Becker

Please contact Anne by email if you have any questions.

keywords: Generic design space exploration framework, software architecture, quality optimization, quality improvement, degrees of freedom