musings

Managing multi-file projects with Scala CLI

In my earlier post on JSON serialization, I built a small API with the uPickle and Cask libraries and included instructions on how to run it with Scala CLI.

The main reason for choosing Scala CLI was that I could put the code and the dependencies in a single file1 (easy to share as a snippet). But code grows over time and files are eventually split up for maintainability.2

I'm going to (very prematurely) do that now and use the opportunity to show how Scala CLI can work for multi-file projects.

Getting started

Here is the code in question:

App.scala

//> using dependency com.lihaoyi::cask:0.10.2

import java.time.LocalDate
import upickle.default._

// Models

given ReadWriter[LocalDate] = readwriter[String].bimap[LocalDate](
  date => date.toString,
  str  => LocalDate.parse(str),
)

case class Release(
    title: String,
    format: String,
    artists: Seq[Artist],
    releaseDate: Option[LocalDate] = None,
) derives ReadWriter

case class Artist(name: String, country: String) derives ReadWriter

// Data

val releases = Seq(
  Release(
    title = "Undoing Ruin",
    format = "album",
    artists = Seq(Artist("Darkest Hour", "USA")),
    releaseDate = Some(LocalDate.parse("2005-06-28")),
  ),
  Release(
    title = "Blackout!",
    format = "album",
    artists = Seq(
      Artist("Method Man", "USA"),
      Artist("Redman", "USA")
    ),
    releaseDate = Some(LocalDate.parse("1999-09-28")),
  ),
)

// Server

object App extends cask.MainRoutes {
  @cask.get("/releases")
  def get() = {
    write(releases)
  }

  initialize()
  println("Server running on localhost:8080")
}

We can run it with Scala CLI's run command:

$ scala-cli run App.scala
Compiling project (Scala 3.6.3, JVM (17))
Compiled project (Scala 3.6.3, JVM (17))
Server running on localhost:8080

And then call the endpoint:

$ curl localhost:8080/releases
[{"title":"Undoing Ruin","format":"album","artists":[{"name":"Darkest Hour","country":"USA"}],"releaseDate":"2005-06-28"},{"title":"Blackout!","format":"album","artists":[{"name":"Method Man","country":"USA"},{"name":"Redman","country":"USA"}],"releaseDate":"1999-09-28"}]

Splitting up the file

First, we'll move the dependencies to their own file.

project.scala

//> using dependency com.lihaoyi::cask:0.10.2

And then do the same for the different concerns or application layers. While we're at it, we'll also add an optional name=<STRING> query parameter to the endpoint, that way we can filter the results.

Model.scala

import java.time.LocalDate
import upickle.default._

given ReadWriter[LocalDate] = readwriter[String].bimap[LocalDate](
  date => date.toString,
  str  => LocalDate.parse(str),
)

case class Release(
    title: String,
    format: String,
    artists: Seq[Artist],
    releaseDate: Option[LocalDate] = None,
) derives ReadWriter

case class Artist(name: String, country: String) derives ReadWriter

Service.scala

import java.time.LocalDate

val releases = Seq(
  Release(
    title = "Undoing Ruin",
    format = "album",
    artists = Seq(Artist("Darkest Hour", "USA")),
    releaseDate = Some(LocalDate.parse("2005-06-28")),
  ),
  Release(
    title = "Blackout!",
    format = "album",
    artists = Seq(
      Artist("Method Man", "USA"),
      Artist("Redman", "USA")
    ),
    releaseDate = Some(LocalDate.parse("1999-09-28")),
  ),
)

class Service() {
  def getReleases(artistName: Option[String] = None): Seq[Release] =
    artistName match {
      case None => releases
      case Some(name) => releases.filter(_.artists.filter(_.name.equalsIgnoreCase(name)).nonEmpty)
    }
}

App.scala

object App extends cask.MainRoutes {
  @cask.get("/releases")
  def getReleases(name: Option[String] = None) = {
    upickle.default.write(getReleases())
  }

  initialize()
  println("Server running on localhost:8080")
}

Running the app

Let's revisit the command we used to run the app earlier:

$ scala-cli run App.scala

Its definition is scala-cli [command] [path_1 ... path_n] where:

What has to change now is the path list. With the original file split up, compiling App.scala on its own doesn't work. The path list must include the new files too:

$ scala-cli run project.scala App.scala Model.scala Service.scala

NOTE: We get an error if project.scala isn't in the path list:

$ scala-cli run App.scala Model.scala Service.scala
Compiling project (Scala 3.6.3, JVM (17))
[error] ./src/main/App.scala:1:20
[error] Not found: cask - did you mean caps?
[error] object App extends cask.MainRoutes {
[error]                    ^^^^
# ....
Error compiling project (Scala 3.6.3, JVM (17))
Compilation failed

The Scala source files won't compile if the dependencies are missing. We centralized them in project.scala, so our run command's path list must always include that file, regardless of which combination of files and directories we want to build and run.

And since the path list consists entirely of files in the root directory, we can substitute the root for the file names and call:

$ scala-cli run .

Adding tests

No good project is complete without tests, so let's learn how to run them with Scala CLI.

First we add the uTest testing library to our dependencies3:

//> using dependency com.lihaoyi::cask:0.10.2
//> using test.dependency com.lihaoyi::utest:0.8.5

NOTE: The using directive for uTest has a test.dependency key instead of dependency because we only need it for tests.

Here's what this would look like in an sbt project:

// build.sbt
libraryDependencies += "com.lihaoyi" %% "cask" % "0.10.2"
libraryDependencies += "com.lihaoyi" %% "utest" % "0.8.5" % Test

Then we write a few tests in a ServiceSpec.scala file:

import utest._

object ServiceSpec extends TestSuite {
  val tests = Tests {
    val service = Service()

    test("returns all releases") {
      val names = Seq("Darkest Hour", "Method Man", "Redman")
      val actual = service.getReleases()
      assert(actual.size == 2)
      assert(actual.flatMap(_.artists.map(_.name)) == names)
    }

    test("returns releases by a given artist") {
      val name = Some("Darkest Hour")
      val actual = service.getReleases(name)
      assert(actual.size == 1)
      assert(actual.head.artists.map(_.name) == name)
    }

    test("returns empty when the artist name is not found") {
      val actual = service.getReleases(Some("abcdefghijklmnop"))
      assert(actual.isEmpty)
    }
  }
}

The method under test is Service.getReleases(artistName: Option[String] = None) and we're verifying that it:

Now we can run the tests with scala-cli test instead of scala-cli run:

$ scala-cli test .
Compiling project (test, Scala 3.6.3, JVM (17))
Compiled project (test, Scala 3.6.3, JVM (17))
-------------------------------- Running Tests --------------------------------
+ ServiceSpec.returns all releases 7ms  
+ ServiceSpec.returns releases by a given artist 1ms  
+ ServiceSpec.returns empty when the artist name is not found 0ms  
Tests: 3, Passed: 3, Failed: 0

Structuring the project

Right now, the files all live at the project root.

Let's create folders to start organizing the code:

The project should look like this after moving the files to the new folders:

root
├── src
│   ├── main
│   │   ├── App.scala
│   │   ├── Model.scala
│   │   └── Service.scala
│   └── test
│       └── ServiceSpec.scala
└── project.scala

What happens if we try the previous command to run the app?

$ scala-cli run .
Server running on localhost:8080

How about the tests?

$ scala-cli test .
-------------------------------- Running Tests --------------------------------
+ ServiceSpec.returns all releases 7ms  
+ ServiceSpec.returns releases by a given artist 1ms  
+ ServiceSpec.returns empty when the artist name is not found 0ms  
Tests: 3, Passed: 3, Failed: 0

They both still work! But why didn't we have to update the path lists this time? The scala-cli path list traversal is recursive, so the contents of src/main/ and src/test/ get picked up along with project.scala when given the root directory.

In fact, it's the only value we can use for the path list that will work:

Choosing an entry point

So far, we've been calling scala-cli with a path list, but without specifying the main class to actually run. And yet that never prevented the application or tests from starting. How does that work?

When scala-cli [command] traverses the path list, it looks for an application entry point based on the command:

Unlike test, run must only find one entry point. This isn't a problem for us because we only have one @main method. A traversal — even starting at the root — has no chance of finding another one.

But if we add a second entry point in src_2/main/App.scala:

root
├── src
│   ├── main
│   │   ├── App.scala            <-- entry point 1
│   │   ├── Model.scala
│   │   └── Service.scala
│   └── test
│       └── ServiceSpec.scala
├── src_2
│   ├── main
│   │   └── App.scala            <-- entry point 2
└── project.scala

Scala CLI complains about finding more than one:

$ scala-cli run .
Compiling project (Scala 3.6.3, JVM (17))
Compiled project (Scala 3.6.3, JVM (17))
[error]  Found several main classes: App, run

We have three options for choosing an entry point:

  1. Passing one with the --main-class flag:
    # entry point 1
    $ scala-cli run project.scala . --main-class App
    
  2. Interactively with the --interactive flag:
    # entry point 1
    $ scala-cli run project.scala . --interactive
    Found several main classes. Which would you like to run?
    [0] App
    [1] run
    0
    
  3. With a path list that excludes every other entry point:
    # entry point 1
    $ scala-cli run project.scala src/
    
    # entry point 2    
    $ scala-cli run project.scala src_2/
    

I like to use the third option when possible because excluding Scala sources from the path list means less code to compile.

Shortening the commands

In the previous sections, we looked at different ways that Scala CLI handles two tasks — run app and run all tests. Since they're the ones that come up most often during development4, we should think about aliasing or shortening their respective commands.

One solution is creating a Makefile5:

.SILENT:
default:
run:
	scala-cli run project.scala src/
test:
	scala-cli test project.scala src/  

With that, running the app is now:

$ make run

And running the tests:

$ make test

Knowing the different ways to call scala-cli is useful for tasks like:

But our Make commands are easier to remember than the original ones! And we can always add to the Makefile as new tasks become more frequent.

Conclusion

Scala CLI is often touted as an alternative to Ammonite, making Scala even more viable for lightweight scripting and CLI tools.

But I wanted to show how structuring your project — particularly the dependencies in project.scala — can make it viable for multi-file projects.

And with multiple entry points, you can create standalone apps that can be run independently and even have their own dependency lists (e.g., src/project.scala and src_2/project.scala). So you'd effectively have a multi-module project that works with Scala CLI — no switching to a larger build tool required.

  1. Using a build tool like sbt or Mill would have required more setup — at the very least, a configuration file separate from the App.scala source.

  2. There's no right answer here and it can depend on factors like your language, frameworks, and code style. The file size might not even be your biggest concern. Maybe for you, maintainability means not having code for account creation, database transactions, and user notifications in the same file.

  3. Another choice would have been fine. The library itself isn't important since our focus is on running tests with Scala CLI, not writing them.

  4. Unless the test suite is slow, at which point developers might take "creative" measures like disabling tests or not running any at all — an unfortunate but logical response when the business values shipping above all else.

  5. See this post for an introduction to Make.

#scala