• 设为首页
  • 点击收藏
  • 手机版
    手机扫一扫访问
    迪恩网络手机版
  • 关注官方公众号
    微信扫一扫关注
    迪恩网络公众号

palantir/gradle-consistent-versions: Compact, constraint-friendly lockfiles for ...

原作者: [db:作者] 来自: 网络 收藏 邀请

开源软件名称(OpenSource Name):

palantir/gradle-consistent-versions

开源软件地址(OpenSource Url):

https://github.com/palantir/gradle-consistent-versions

开源编程语言(OpenSource Language):

Java 64.3%

开源软件介绍(OpenSource Introduction):

Autorelease

com.palantir.consistent-versions Download

A gradle plugin to ensure your dependency versions are consistent across all subprojects, without requiring you to hunt down and force every single conflicting transitive dependency.

Direct dependencies are specified in a top level versions.props file and then the plugin relies on Gradle constraints to figure out sensible versions for all transitive dependencies - finally the whole transitive graph is captured in a compact versions.lock file.

  1. Apply the plugin (root project only):

    plugins {
        id "com.palantir.consistent-versions" version "<current version>"
    }

    You can find the current version under releases.

  2. In one of your build.gradle files, define a versionless dependency on some jar:

    apply plugin: 'java'
    
    dependencies {
        implementation 'com.squareup.okhttp3:okhttp'
    }
  3. Create a versions.props file and provide a version number for the jar you just added:

    com.squareup.okhttp3:okhttp = 3.12.0
    
  4. Run ./gradlew --write-locks and see your versions.lock file be automatically created. This file should be checked into your repo:

    # Run ./gradlew --write-locks to regenerate this file
    com.squareup.okhttp3:okhttp:3.12.0 (1 constraints: 38053b3b)
    com.squareup.okio:okio:1.15.0 (1 constraints: 810cbb09)

Contents

  1. Motivation
    1. An evolution of nebula.dependency-recommender
  2. Concepts
    1. versions.props: lower bounds for dependencies
    2. versions.lock: compact representation of your prod classpath
    3. ./gradlew why
    4. ./gradlew checkUnusedConstraints
    5. getVersion
    6. BOMs
    7. Specifying exact versions
    8. Downgrading things
    9. Common workflow: SLF4J
    10. Common workflow: dependencySubstitution
    11. Common workflow: internal test utility projects
    12. Resolving dependencies at configuration time is banned
    13. Known limitation: root project must have a unique name
    14. Scala
  3. Migration
    1. How to make this work with Baseline
    2. dependencyRecommendations.getRecommendedVersion -> getVersion
  4. Technical explanation
    1. Are these vanilla Gradle lockfiles?
    2. Conflict-safe lock files

Motivation

Many languages have arrived at a similar workflow for dependency management: direct dependencies are specified in some top level file and then the build tool figures out a sensible version for all the transitive dependencies. The whole dependency graph is then written out to some lock file. We asked ourselves, why doesn't this exist for Java?

  • JavaScript repos using yarn define dependencies in a package.json and then yarn auto-generates a yarn.lock file.
  • Rust repos using cargo define direct dependencies in a cargo.toml file and have cargo.lock auto-generated.
  • Go repos using dep define dependencies in a Gopkg.toml file and have Gopkg.lock auto-generated.
  • Ruby repos using bundler define dependencies in a Gemfile file and have Gemfile.lock auto-generated.

Specifically this plugin delivers:

  1. one version per library - When you have many Gradle subprojects, it can be frustrating to have lots of different versions of the same library floating around on your classpath. You usually just want one version of Jackson, one version of Guava etc. (failOnVersionConflict() does the job, but it comes with some significant downsides - see below). With gradle-consistent-versions, dependencies across all your subprojects have a sensible version provided.
  2. better visibility into dependency changes - Small changes to your requested dependencies can have cascading effects on your transitive graph. For example, you might find that a minor bump of some library brings in 30 new jars, or affects the versions of your other dependencies. With gradle-consistent-versions, it's easy to spot these changes in your versions.lock.

An evolution of nebula.dependency-recommender

nebula.dependency-recommender pioneered the idea of 'versionless dependencies', where gradle files just declare dependencies using compile "group:name" and then versions are declared separately (e.g. in a versions.props file). Using failOnVersionConflict() and nebula.dependency-recommender's OverrideTransitives ensures there's only one version of each jar on the classpath.

nebula.dependency-recommender forces all your dependencies, which overrides all version information that libraries themselves provide.

Unfortunately, failOnVersionConflict means developers often pick conflict resolution versions out of thin air, without knowledge of the actual requested ranges. This is dangerous because users may unwittingly pick versions that actually violate dependency constraints and may break at runtime, resulting in runtime errors suchs as ClassNotFoundException, NoSuchMethodException etc

Concepts

versions.props: lower bounds for dependencies

Specify versions for your direct dependencies in a single root-level versions.props file. Think of these versions as the minimum versions your project requires.

com.fasterxml.jackson.*:jackson-* = 2.9.6
com.google.guava:guava = 21.0
com.squareup.okhttp3:okhttp = 3.12.0
junit:junit = 4.12
org.assertj:* = 3.10.0

The * notation ensures that every matching jar will have the same version - they will be aligned1 using a virtual platform.

Note that this does not force okhttp to exactly 3.12.0, it just declares that your project requires at least 3.12.0. If something else in your transitive graph needs a newer version, Gradle will happily select this. See below for how to downgrade something if you really know what you're doing.

[1]: If multiple lines from versions.props match a particular jar, the most specific one will be chosen (the one with the most characters being different from *). This has the side effect that a line referring specifically to a jar is independent, and that jar's version never gets aligned to the versions of other jars, even if there are other lines containing * which would otherwise match that jar.

versions.lock: compact representation of your prod classpath

When you run ./gradlew --write-locks, the plugin will automatically write a new file: versions.lock which contains a version for every single one of your transitive dependencies.

Notably, this lockfile is a compact representation of your dependency graph as it just has one line per dependency (unlike nebula lock files which spanned thousands of lines).

javax.annotation:javax.annotation-api:1.4.0 (3 constraints: 1a310f90)
javax.inject:javax.inject:1 (2 constraints: d614a0ab)
javax.servlet:javax.servlet-api:3.1.0 (1 constraints: 830dcc28)
javax.validation:validation-api:1.1.0.Final (3 constraints: dc393f20)
javax.ws.rs:javax.ws.rs-api:2.0.1 (8 constraints: 7e9ce067)

[Test dependencies]
cglib:cglib-nodep:3.1 (1 constraints: 2a0e1330)
com.github.zafarkhaja:java-semver:0.9.0 (1 constraints: c315c0d2)
com.jayway.awaitility:awaitility:1.6.5 (1 constraints: c615c1d2)

The lockfile sources production dependencies from the compileClasspath and runtimeClasspath configurations, and test dependencies from the compile/runtime classpaths of any source set that ends in test (e.g. test, integrationTest, eteTest).

There is a verifyLocks task (automatically run as part of check) that will ensure versions.lock is still consistent with the current dependencies.

./gradlew why

To understand why a particular version in your lockfile has been chosen, run ./gradlew why --hash a60c3ce8 to expand the constraints:

> Task :why
com.fasterxml.jackson.core:jackson-databind:2.9.8
        com.fasterxml.jackson.module:jackson-module-jaxb-annotations -> 2.9.8
        com.netflix.feign:feign-jackson -> 2.6.4
        com.palantir.config.crypto:encrypted-config-value -> 2.6.1
        com.palantir.config.crypto:encrypted-config-value-module -> 2.6.1

This is effectively just a more concise version of dependencyInsight:

./gradlew  dependencyInsight --configuration unifiedClasspath --dependency jackson-databind

You can check multiple dependencies at once by passing multiple comma-delimited hash values, e.g. ./gradlew why --hash a60c3ce8,400d4d2a.

./gradlew checkUnusedConstraints

checkUnusedConstraints prevents unnecessary constraints from accruing in your versions.props file. Run ./gradlew checkUnusedConstraints --fix to automatically remove any unused constraints from your props file.

getVersion

If you want to use the resolved version of some dependency elsewhere in your Gradle files, gradle-consistent-versions offers the getVersion(group, name, [configuration]) convenience function. For example:

task demo {
    doLast {
        println "We chose guava " + getVersion('com.google.guava', 'guava')
    }
}

This function may not be invoked at Gradle Configuration time as it involves resolving dependencies. Put it inside a closure or provider to ensure it is only invoked at Execution time.

By default, this function resolve the unifiedClasspath configuration to supply a version, but you can always supply a different configuration if you want to:

task printSparkVersion {
    doLast {
        println "Using spark version: " + getVersion('org.apache.spark', 'spark-sql_2.11', configurations.spark)
    }
}

BOMs

Gradle has first-class support for sourcing version constraints from published BOMs so they work fine with gradle-consistent-versions:

allprojects {
    apply plugin: 'java-base'
    dependencies {
        rootConfiguration platform('com.foo.bar:your-bom')
    }
}

Make sure you apply BOMs within an allprojects closure, as gradle-consistent-versions must be able to unify constraints from all subprojects.

Note: java-base is necessary, even on projects that don't have java source code, otherwise gradle will silently interpret the platform(...) dependency as if it was a normal library dependency, and will not import the constraints from that BOM.

Specifying exact versions

The preferred way to control your dependency graph is using dependency constraints on gradle-consistent-versions' rootConfiguration. For example:

dependencies {
    constraints {
        rootConfiguration 'org.conscrypt:conscrypt-openjdk-uber', {
            version { strictly '1.4.1' }
            because '1.4.2 requires newer glibc than available on Centos6'
        }

        rootConfiguration 'io.dropwizard.metrics:metrics-core', {
            version { strictly '[3, 4[' }
            because "Spark still uses 3.X, which can't co-exist with 4.X"
        }
    }
}

Gradle will fail if something in your dependency graph is unable to satisfy these strictly constraints. This is desirable because nothing is forced in your transitive graph.

Downgrading things

If you discover a bug in some library on your classpath, the recommended approach is to use dependencyInsight to figure out why that version is on your classpath in the first place and then downgrade things until that library is no longer brought in. Once the dependency is gone, you can specify a rootConfiguration constraint to make sure it doesn't come back (see above).

./gradlew dependencyInsight --configuration unifiedClasspath --dependency retrofit

Occasionally however, downgrading things like this is not feasible and you just want to force a particular transitive dependency. This is dangerous because something in your transitive graph clearly compiled against this library and might be relying on methods only present in the newer version, so forcing down may result in NoSuchMethodErrors at runtime on certain codepaths.

allprojects {
    configurations.all {
        resolutionStrategy {
            force 'com.squareup.retrofit2:retrofit:2.4.0'
        }
    }
}

Common workflow: SLF4J

Developers usually want just one SLF4J implementation on the classpath. If some of your dependencies rely on their own logging implementations (e.g. commons-logging or log4j), you can use the following snippet to ensure that all logging will go through SLF4J.

allprojects {
    dependencies {
        modules {
            module('commons-logging:commons-logging') {
                replacedBy('org.slf4j:jcl-over-slf4j', 'slf4j allows us supply our own implementation')
            }
            module('log4j:log4j') {
                replacedBy('org.slf4j:log4j-over-slf4j', 'slf4j allows us supply our own implementation')
            }
        }
    }
}

Common workflow: dependencySubstitution

We've seen the following error when using dependencySubstitution:

> Could not find method module() for arguments [org.glassfish.hk2.external:javax.inject] on configuration ':my-project:subprojectUnifiedClasspathCopy' of type org.gradle.api.internal.artifacts.configurations.DefaultConfiguration.

Adding explicit it calls works around this error:

 configurations.configureEach {
     resolutionStrategy.dependencySubstitution {
-        substitute module('org.glassfish.hk2.external:javax.inject') with module('javax.inject:javax.inject:1')
+        it.substitute it.module('org.glassfish.hk2.external:javax.inject') with it.module('javax.inject:javax.inject:1')
     }
 }

Common workflow: internal test utility projects

Sometimes, devs have multiple test projects (unit tests, integration tests) that use a subset of common test classes.

* :foo
   \--- source set 'test'
         \--- :foo-test-common
* :foo-integration
   \--- source set 'integrationTest'
         \--- :foo-test-common
* :foo-test-common
   \--- source set 'main'

In this case, we'd like to prevent GCV from locking :foo-test-common's main source set to production dependencies, and instead treat the entire project as test dependencies. We can do this via the following snippet:

# foo-test-common/build.gradle
apply plugin: 'java'

versionsLock {
    testProject()
}

Resolving dependencies at configuration time is banned

In order for this plugin to function, we must be able to guarantee that no dependencies are resolved at configuration time. Gradle already recommends this but gradle-consistent-versions enforces it.

In many cases, it's just a matter of using a closure or a provider, for example:


鲜花

握手

雷人

路过

鸡蛋
该文章已有0人参与评论

请发表评论

全部评论

专题导读
热门推荐
阅读排行榜

扫描微信二维码

查看手机版网站

随时了解更新最新资讯

139-2527-9053

在线客服(服务时间 9:00~18:00)

在线QQ客服
地址:深圳市南山区西丽大学城创智工业园
电邮:jeky_zhao#qq.com
移动电话:139-2527-9053

Powered by 互联科技 X3.4© 2001-2213 极客世界.|Sitemap