Probe specification

This page collects what has been designed for the probe specification so far

Motivation

The collection of performance data in the PCM is currently hard coded in the analyses. For example, the simulation collects the raw response time measurements for each simulated service call. In the future, it should be configurable which data is collected. For this, we need Probes which tell the analyses which data to collect. The resulting probe specification framework should be compatible to the SensorFramework and independent from the PCM (as the SensorFramework itself is).

Basic Idea

Probes annotate model elements (EObjects). Whenever the analyses encounters an model element annotated with a probe, it should collect data. There will be PCM specific implementation for the abstract concepts.

Abschlusspräsentation ProbeSpec Praktikumsgruppe

Types of Probes

Probes sind Abstraktum; Spezifische Probes sind Erben und legen erhobene Daten (Time, Date, ...) fest. Pro Probe sollten zu erhebende Daten spezifiziert werden können: obliegt spezifischen Probes

Unterschiedliche Informationen die von Probes gemessen werden:

CurrentTimeProbe
SEFFParameterProbe
DemandProbe
UtilisationProbe
...

If Probes are attached to for example SEFF actions, we need to distinguish when the Probe is activated / fired: before the action or after the action.

Types of Measurements (Calculators)

Whereas probes could specify what to measure by their type, a concept Calculator could be used to define whether a certain snapshot property (current number of threads on CU -> utilisation) or a difference (timespan for response time) should be measured. In theory, measuring the properties (event time, request id) for a service start and a service completion would be enough to deduct the time span. However, it might be much more convenient to more explicitly support time spans.

On the one hand, Calculators should be able to collect rather complex things. On the other hand, users have to specify them at the user interface. There will be predefined Calculators that collect a certain data (and may be configurable), and the user just chooses some together with a number of probes. Additionally, users can implement their own custom Calculators.

See framework documentation for more details on Calculators.

Additionally, it could be specified whether aggregations (sliding mean) or certain ranges (only after the 1000th call (warm-up)) should be measured. Maybe even more complex conditions for ranges could be given, e.g. the detection that warm-up has finished if the mean value does not change so much any more. This is separated from the Calculators and realised in the Pipe&Filter framework.

Scenarios

What needs to be supported by the Probe Specification

Collected Performance Metrics and Data

All data that can be collected in SimuCom so far
- Resource utilisation
- Waiting times at resources
- Demands issued to resources
- end-to-end response time of services
- ...
Demands by components to resources (could also be done by dependency solver alone, but this would not take into account the actually drawn samples from random variables). With this information, allocation decisions could be supported.
Utilisation of components? Does this make sense? Maybe the number of users / threads currently active in one component. But what questions could be answered?
Aggregation and ranges need to be specified somewhere. This might be in the Probes(?) or rather in a separate Recorder definition. For example, for a certain metric, we might not want to store all values, but just the sliding mean. This has to be configurable. --> Introduce Recorders.
- Müssen Recorder und Probe von einem Typ sein oder können sie auch so zusammenpassen? TODO: Scenarien überlegen.

Implementation

Code location, Javadoc documentation

Probe metamodel that allows the attaching of probes to model elements to specify where and what to measure. It decorates the models (e.g. a PCM model): RSA Modelle. The documentation of the concepts is found in the respective implementation class
- A generic part independent of PCM.
- Maybe a PCM-specific part that has Probes specific to PCM metamodel elements (TODO)
- Probably the transformation looks up the probes and generates probe calls at e.g. each method call. This would mean that the probe definition cannot be altered during the simulation run. (TODO)
Probe framework that handles the storage of probe data in the sensorframework and offers convenience methods (such as request id stack trace stuff):
- Generic independent of the PCM
- A SimuCom specific part (TODO)
Pipes&Filters framework that allows filtering and saving the collected data
- A generic part
- An EDP2 specific part
- Optionally a LQNS specific part that retrieves the requested data from the LQNS results if available (can only support a subset) and puts it in the sensorframework / Experiment Data Persistency & Presentation (EDP2). Maybe it would be even more appropriate to just store all LQNS results in the EDP2, ignoring the probes. (TODO)
SimuCom adjustment: The simulation has to notify the probe framework if a probe is encountered in the models. Here, it has to provide the right data depending on the probe or allow the probe framework access to its own data. (TODO)
Probe specification support: A language to conveniently define probes for a certain model and a GUI support for it (EMF models available, Editors TODO).

Specification of Probes

We do not want to create all Probes manually. There needs to be a graphical user interface that allows to create a number of probes based on a "Probe Specification Language" or the like. For example, this language should support a statement like "for all component services, measure the time from the call of the service to its completion" (the current service sensor behaviour). This is not realised yet.

Zukünftige Anforderungen "Wishlist"

Request IDs
- Request IDs werden von extern in Probes hinein gegeben
- Bei Forks wird Stack mit Request IDs aufgebaut
  - Scenario: A stop probe can be reaches multiple time by a request that has only reached the start probe once (because of the fork). Then, we want to take all measurements.
  - Sensoren haben Menge von Probes und können Request Stack auswerten und nur reagieren, wenn der richtige Stackframe / Teil-Stackframe vorliegt (that matches their already encountered start probes).
- Be general: No concrete
- The probe specification framework should offer a method like "attachUUIDStackTrace" and similar ones.
Unterstützung verteilter Messungen
Kein Konflikt mit ApplicationServern (Messungen aus Programmen, die in Application Servern ausgeführt werden sollten möglich sein)
Umgang mit Parallelität / Atomaritätswahrung der Implementierung des Sensorframeworks
Identification of calls via data (in producer/consumer systems, for example, where there is no unique request id, but the request is identified via the consumed message). Probably never done for PCM, but maybe for other analyses.
- Use the Stacktrace for Datenfluss: ID bei Probe-Daten anhängen können (generieren eher in ein weiteres Projekt ziehen)

Technical

Global probes: measurements that cross operating system boundaries
Decision of 11.11.2008: Probes or request ids do not have to be attached to data in the PCM, for example to model producer-consumer systems. We do not model the data item s explicitly enough. However, this view might change with internal state and message-oriented middleware stuff of Chris. However, in general, the probes should also allow to collect data-related measurement. Thus, attach the general Probe and the general id to EObject or the like.
There must be the possibility to activate or deactivate Probes. You do not want to delete Probes only if you do not want their results in one run, as you have to create them anew afterwards.

Keyword: ProbeSpecification, ProbeSpec, Probes, Measurements, EDP2, Pipes, Filters, Pipe&Filter, Pipes-and-Filter