Talaiot is an extensible library targeting teams using the Gradle Build System.
It records build/task duration helping to understand problems of the build and detecting bottlenecks. For every record, it will add additional information defined by default or custom metrics.
Talaiot is compatible with different systems like InfluxDb, Elasticsearch or RethinkDb. You need to use a Plugin to work with Talaiot.
You can use the standard plugin, including all the functionality, or if you have a specific requirement with a individual plugin.
Current available plugins:
Plugin
Description
standard
Contains all the available publishers listed below
base
Talaiot core functionality with Json, Output and Timeline publishers
elasticsearch
Talaiot core functionality with Elasticsearch publisher
graph
Talaiot core functionality with Graph publisher
influxdb
Talaiot core functionality with Influxdb publisher
pushgateway
Talaiot core functionality with Pushgateway publisher
rethinkdb
Talaiot core functionality with Rethinkdb publisher
Once you have Talaiot integrated you can create dashboards using the build information stored:
What is Talaiot?
"... while some certainly had a defensive purpose, the use of others is not clearly understood. Some believe them to have served the purpose of lookout or signalling towers..."
Generation of unique identifier for each execution(disabled by default)
publishers
Configuration to define where to submit the information of the build
metrics
Additional information tracked during the execution of the task
filter
Rules to filter the build or the tasks to be reported
Metrics
We can include extra information on the build and task tracked data during the build. This information will be added to the default metrics defined.
talaiot {
metrics {
// You can add your own custom Metric objects:
customMetrics(
MyCustomMetric(),
// Including some of the provided metrics, individually.
HostnameMetric()
)
// Or define build or task metrics directly:
customBuildMetrics(
"kotlinVersion" to $kotlinVersion,
"javaVersion" to $javaVersion
)
customTaskMetrics(
"customProperty" to $value
)
}
}
The Publisher configuration will change depending on the type of plugin you are using. Standard Plugin provides
all the different publisher configurations, for example:
If you need to include custom metrics as tags, you need to use the type Custom
Included in: io.github.cdsap.talaiot and io.github.cdsap.talaiot.plugin.influxdb plugins.
InfluxDb2Publisher
Talaiot will send to the InfluxDb (Flux) server defined in the configuration the values collected during the execution
Property
Description
token
Influx access token
url
Url of the InfluxDb Server
taskMetricName
Name of the metric used for specific task in the execution
buildMetricName
Name of the metric used for the overall information of the build in the execution
org
Organization name
bucket
Name of bucket
publishBuildMetrics
Publish build metrics of the publisher, true by default
publishTaskMetrics
Publish tasks metrics of the publisher, true by default
buildTags
Collection of BuildMetrics used as tags
taskTags
Collection of TaskMetrics used as tags
Included in: io.github.cdsap.talaiot and io.github.cdsap.talaiot.plugin.influxdb2 plugins.
RetentionPolicyConfiguration
Retention Policy (RP) describes how long InfluxDB keeps data, how many copies of the data to store in the cluster,
and the time range covered by shard groups. RPs are unique per database and along with the measurement and tag set define a series.
Since version 1.0.0 we are including by default RP in all the operations included in the publisher. The RetentionPolicyConfiguration includes:
Property
Description
name
name of the retentionPolicy(rp). Default rpTalaiot
duration
duration of the rp. Default 30d
shardDuration
the shardDuration. Default 30m
replicationFactor
the replicationFactor of the rp. Default 2
isDefault
if the rp is the default rp for the database or not. Default false
Export the task dependency graph in png format. See Graphviz
This new category of publishers does not require constantly evaluating the builds, that's why there is an extra
parameter configuration in the Publisher to ignore the execution unless there is some property enabled. Typical use case is
use this publisher and collect the files on CI.
The output will be found "${project.rootDir}/talaiot:
Example:
Included in: io.github.cdsap.talaiot and io.github.cdsap.talaiot.plugin.graph plugins.
PushGatewayPublisher
Talaiot will send to the PushGateway server defined in the configuration the values collected during the execution.
Property
Description
url
Url of the PushGateway Server
taskJobName
Name of the job required for the tasks metrics to be exported to Prometheus
buildJobName
Name of the job required for the build metrics to be exported to Prometheus
publishBuildMetrics
Publish build metrics of the publisher, true by default
publishTaskMetrics
Publish tasks metrics of the publisher, true by default
Included in: io.github.cdsap.talaiot and io.github.cdsap.talaiot.plugin.pushgateway plugins.
JsonPublisher
Talaiot will Publish the results of the build with a json format .
publishers {
jsonPublisher = true
}
Included in: io.github.cdsap.talaiot and io.github.cdsap.talaiot.plugin.base plugins.
TimelinePublisher
Talaiot will create a PNG file with the detailed information in chronological order by task of the execution
in the different workers.
publishers {
timelinePublisher = true
}
Included in: io.github.cdsap.talaiot and io.github.cdsap.talaiot.plugin.base plugins.
ElasticSearchPublisher
Talaiot will send to the ElasticSearch server defined in the configuration the values collected for tasks and build metrics during the execution
in the different workers.
Property
Description
url
ElasticSearch server
taskIndexName
Name for the index used to report tasks metrics
buildIndexName
Name for the index used to report build metrics
publishBuildMetrics
Publish build metrics of the publisher, true by default
publishTaskMetrics
Publish tasks metrics of the publisher, true by default
In this example we are using InfluxDbPublisher to report build metrics and ElasticSearchPublisher to report task metrics.
Included in: io.github.cdsap.talaiot plugin.
RethinkDbPublisher
Talaiot will send to the RethinkDb server defined in the configuration the values collected during the execution
Property
Description
dbName
Name of the database
url
Url of the RethinkDb Server
taskTableName
Name of the table used to track tasks information
buildTableName
Name of the table used to track the build information
username
username which is used to authorize against the RethinkDb instance (optional)
password
password for the username which is used to authorize against the RethinkDb (optional)
publishBuildMetrics
Publish build metrics of the publisher, true by default
publishTaskMetrics
Publish tasks metrics of the publisher, true by default
Included in: io.github.cdsap.talaiot and io.github.cdsap.talaiot.plugin.rethinkdb plugins.
Custom Publishers
Talaiot allows using custom publishers defined by the requirements of your environment, in case you are using another implementation.
talaiot {
publishers {
// You can define one or more custom publishers:
customPublishers(
MyCustomPublisher()
)
}
}
Read more about it in the Publishers wiki page
In you are not using additional plugins we recommend to use io.github.cdsap.talaiot.plugin.base.
Sample
Folder sample contains a Gradle project using Talaiot:
cd sample
./gradlew assemble
The project includes examples creating custom publishers, defining metrics using different publishers.
Example: Analyzing Data provided by Talaiot
Docker, InfluxDb and Grafana
To have a quick setup to see the possibilities of Talaiot we are providing a Docker image to setup a Grafana + InfluxDb instances(based on this great repo).
Additionally, the Docker image is creating a default database, a provisioned dashboard and the default datasource for InfluxDb.
The source is here:
To see Talaiot in action, you need to populate the data. We are providing a script to populate data based in the sample project included in the repository.
You can execute the script:
bash scripts/populate.sh
The script will download the repository and with the help of Gradle Profiler(https://github.com/gradle/gradle-profiler)
will trigger number of builds defined in the scenario file:
请发表评论