sbt Library Dependencies et al

sbt provides Library Dependencies as a way to configure external dependencies to a project. This has implications on building, publishing and/or dockerizing outputs.

Hope to provide the differences between managedDependencies and unmanagedDependencies and also on how this can tweaked for certain packaging units.

Usually multi-project based sbt contains the following kinds of dependencies

• 3rd party library dependency (commonly resolved using maven or ivy)
• hard wired jar filepath (less commonly used and is found in situations where the dependency is not managed by a maven like repository).
• sub-project based dependency using dependsOn

The first one is commonly referred as managedDependencies and the later two is referred as unmanagedDependencies .

Managed Dependencies

Managed dependencies usually provided using libraryDependencies and are typically chained with multiple of them.

libraryDependencies += organization %% moduleName % version

With managed dependencies, there are number of ways these dependencies can be sent to output.

Provided

Provided is a way dependencies are not transitive added during publish step.

libraryDependencies += "org.scalatest" %% "scalatest" % "3.1.0"     % "test" % Provided

exclude()

Its also possible to exclude selective transitive dependencies using exclude() . And these are excluded during publish step.

libraryDependencies += "log4j" % "log4j" % "1.2.15" exclude("javax.jms", "jms")

dependencyOverride

dependencyOverride is a way to override certain transitive dependencies that are resolved to a project.

dependencyOverrides += "com.fasterxml.jackson.core" % "jackson-core" % "3.7.0-M11"

This way jackson-core would be resolved to 3.7.0-M11 even though a dependency brings a newer version of this dependency.

Unmanaged Dependencies

While, unmanaged dependencies are provided using a lib directory as a convention. This can be overridden using

unmanagedBase := baseDirectory.value / "custom_lib"

Docker package

These days microservices are typically published as docker images. And this requires dependency on an sbt plugin

addSbtPlugin("se.marcuslonnberg" % "sbt-docker" % "1.8.2")

A typical docker configuration to generate Dockerfile may be based on

docker / dockerfile := {
  val jarFile: File                = (Compile / packageBin / sbt.Keys.`package`).value
  val dependencies                 = (Compile / dependencyClasspathAsJars).value
  val jarTarget                    = s"/app/${jarFile.getName}"
  def addLibPath(files: Seq[File]) = files.map("/app/" + _.getName)
  val classpathString              = (Seq(jarTarget, "/app/config") ++ addLibPath(dependencies.files)).mkString(":")

  val banner =
    """echo "Using JAVA_OPTS: ${JAVA_OPTS}"
      |echo "Using args $@"
      |""".stripMargin
  val javaEnv = "java -Dfile.encoding=UTF-8 $JAVA_OPTS"
  val args    = """"$@""""

  val content =
    s"""#!/bin/bash
       |$banner
       |echo '${name.value} - v${version.value}'
       |echo 'Built on ${LocalDateTime.now()} - ${git.gitHeadCommit.value.getOrElse("No head commit")}'
       |$javaEnv -cp $classpathString $mainclass $args
       |""".stripMargin

  val programName = "my-app"

  val startScript = file(s"""/tmp/$programName""")
  IO.write(startScript, content, append = false)

  new sbtdocker.Dockerfile {
    from("amazoncorretto:11")
    add(dependencies.files, "/app/")
    add(startScript, s"""/$programName""")
    run("chmod", "+x", s"""/$programName""")
    add(jarFile, jarTarget)
    run("mkdir", "-p", "/app/config")
    run("touch", "/app/config/application.conf")
    env("JAVA_OPTS", "-Xms8g")
    env("JAVA_HOME", "/usr/lib/jvm/jre")
    entryPoint(s"""/$programName""")
  }
}

This can be verified using

sbt 'project sub-project-1' docker dockerPush

Note:

(1) the -Dfile.encoding=UTF-8 ensures that strings are processed using utf-8.

(2) Using Built on would ensure to verify the image build timestamp and/or version stamp of different dependencies.

(3) Using git.gitHeadCommit would get the git-commit HEAD SHA1 used for this build.

(4) The classpathString would enable customize classpath required for application start.

(5) The dependencyClasspathAsJars would normalize entries from managedClasspath as jar files as labelled as well as from dependsOn from a multi-project setup.

Publishing to Maven

Typically if one wants to publish to maven using artifactory

ThisBuild / resolvers += "Artifactory-release" at "https://test1.jfrog.io/artifactory/libs-release/"
ThisBuild / resolvers += "Artifactory-snapshot" at "https://test1.jfrog.io/artifactory/libs-snapshot/"
ThisBuild / credentials += Credentials(Path.userHome / ".sbt" / "credentials")

This can be verified using

sbt 'project sub-project-1' publishLocal publish

Tip

In the case, if one has multiple non-mono repository based dependencies, its easier to verify dependency integrity using publishLocal and by providing a local resolver using

resolvers += Resolver.file("local-ivy", file(Path.userHome.absolutePath + "/.ivy2/local"))(Resolver.ivyStylePatterns)

Thank you for reading this post and allowing me to learn to do this as well. Comments / suggestions welcome.

Source: https://www.scala-sbt.org/1.x/docs/Library-Dependencies.html