Who's using com-lihaoyi projects, where, which, for what?

[lihaoyi]

Hi all! I wanted to try and gather some feedback from the com-lihaoyi user base on how they are using these tools and libraries. If you use any of the com-lihaoyi projects (Mill, Ammonite, Cask, uPickle, os-lib, ...), could you please respond with:

What company do you work at (if any)?

How many engineers in the org? (approx is fine)

Which com-lihaoyi projects?

What you like about the projects you use?

What don't you like, either about the projects you use or the projects you don't? (confusing APIs, docs, missing functionality, etc.)

OPTIONAL: If you're open to chat, drop an email to [email protected] and I can reach out with more questions and discussions offline.

I'm trying to get a better sense of the mood of the community using the com-lihaoyi projects, and learn about how they fit into the larger Scala and software engineering ecosystem. So if you have a moment please reply and let us know how things are going, and drop me an email if you are OK with me reaching out with more questions :)

[mgilbertnz]

Hi!

What company do you work at (if any)?
Advice Link. A New Zealand based CRM SaaS provider.

How many engineers in the org? (approx is fine)
4

Which com-lihaoyi projects?
uPickle
requests
sourcecode

What you like about the projects you use?

• Simple APIs
• uPickle performance is very nice
• source code is great for logging "where" in the code something came from. Was excellent when we were running 2 systems in parallel and we were able to point to where in the code results were differing in our logs.

What don't you like, either about the projects you use or the projects you don't? (confusing APIs, docs, missing functionality, etc.)

Not looking to complain about excellent libraries... If I had a magic wand I would somehow do the following..

uPickle

• Have an option to default Option's to None if keys not found rather than having to specify default values.
• I think there is weird deserialization of Ints where null becomes 0.

[fujohnwang]

What company do you work at (if any)?

consulting and tooling software at Maniland China

How many engineers in the org? (approx is fine)

1, myself

Which com-lihaoyi projects?

1. ammonite
2. requests
3. os-libs
4. mill

What you like about the projects you use?

simple, easy API design

What don't you like, either about the projects you use or the projects you don't? (confusing APIs, docs, missing functionality, etc.)

In fact, I had not used any scala libraries anymore, but Java ones.

Since I chose to stay with scala2.12 and focus on what to do instead of what to use or how to to do, stability and compatability is what I prefer most.

[objektwerks]

What company do you work at? --- Objektwerks
How many engineers in the org? --- 1
Which com-lihaoyi projects? --- ujson, upickle, utest, cask, os-lib, requests
What do you like about the projects you use? --- simple, concise, well-documented
What don't you like, either about the projects you use or the projects you don't? --- Switched from uPickle to Jsoniter due to Scala 3 performance issues. Switched from Cask to Helidon SE for virtual thread support and zero annotations. ;) But open to using both in the future.

[julienrf]

What company do you work at (if any)?

Soaringmeteo (non-profit organization)

How many engineers in the org? (approx is fine)

1-5

Which com-lihaoyi projects?

os-lib, requests.

What you like about the projects you use?

No ceremony to set up. Add the dependency to your build and use them. Cherry on the cake: you usually don’t need much imports as well because package names are short and simple (os, requests, ujson).

What don't you like, either about the projects you use or the projects you don't? (confusing APIs, docs, missing functionality, etc.)

• throwing exceptions is not convenient in Scala since exceptions are unchecked, so I would prefer a more explicit way to report failures (e.g., a systematic try prefix in method names that throw, a bold statement in the documentation, or maybe returning Try instead of throwing — we can always add .get if we don’t care about an exception to be thrown)
• reliance on implicit conversions or “magnet pattern” make the API and compilation errors difficult to understand (I’ve seen more than once people being confused by ujson.Arr(Seq(ujson.Num(42))) producing [[42]] instead of [42] because Seq is implicitly converted into a JSON array).

[davesmith00000]

What company do you work at (if any)? / How many engineers in the org? (approx is fine)

Just responding for myself (OSS), however, I've seen com-lihaoyi libs used at every company I've worked at over the last ...decade? Generally I've seen them used for glue code and scripts.

Which com-lihaoyi projects?

• OS-Lib is a personal favourite, I use it a lot, and I've seen it used commercially quite a bit. Can't fault it.
• Mill is my preferred build tool for my own projects, sbt is still the incumbent at $WORK / legacy projects.
• Some use of uJson, uTest, and Requests.

What you like about the projects you use?

Generally I love the straight-forward practically it of the libraries in this ecosystem.

There is no better way to do bash-like scripting stuff than with os-lib. Its simple and gives great confidence. (I wrote a blog post on it!)

Mill deserves much wider usage. I love the fact that you can learn it in a weekend, that it is straight up Scala, and has a highly regular structure. Also writing plugins / contrib extensions for Mill is a joyful experience. The source code is also impressively readable, and I have occasionally worked out how to do stuff by delving into it. (I'm using it in this video because it was far more convenient than the alternatives!)

I used to use uTest extensively too, but I had to switch because I was a very early Scala 3 (pre-release) adopter. uTest wasn't there yet so I had to move and haven't quite bothered myself to go back. But I did love it. It was a breath of fresh air after the DSL heavy libs like ScalaTest and Specs2. These days, uTest has MUnit as competition in that space, but one thing uTest can do that MUnit can't (I think...) is hierarchically nested test - I used to use that a lot.

What don't you like, either about the projects you use or the projects you don't? (confusing APIs, docs, missing functionality, etc.)

I used to use uPickle (a few years ago!) but stopped because I found the encoding choices unexpected. Maybe it has moved on, but things like encoding Options as arrays is technically valid, but unexpected.

My only grumble with Mill (which again, is generally awesome) is the CLI experience. Again, perhaps some of this has been addressed, but there are/were a few footguns that sometimes I could only solve by asking people on Discord. Some examples:

• test runs the tests on the tests module, when in order to be strictly regular, it should be test.test - seems fine but testOnly has to be run test.testOnly IIRC. Took me ages to work out what I was doing wrong.
• I really really dislike launch scripts (I want to install the app and it works with no messing about), yet, millw is the right way to run Mill at the moment (mostly due to the automatic mill version switching). It would be good to bring the two together somehow and make the script transparent to the user.
• Mill doesn't have out-of-the-box autocompletions like sbt (in the console...), nor does it have an equivalent to sbt-welcome. I know there is a completions script somewhere, but like millw, I think it should be there by default. I find command discovery on the CLI a real usability impediment. I'm having a bad time of it if I have to resort to running resolve.
• The need to use single quotes on ZSH in some cases is also not obvious, but may be unavoidable.

Hope that helps. I really appreciate the existence of this ecosystem.

[He-Pin]

What company do you work at (if any)?
Alibaba's Taobao

How many engineers in the org? (approx is fine)
Not sure:(, maybe 1000+

Which com-lihaoyi projects?
Fastaprse

What you like about the projects you use?

1. Using fastparse to implement a JsonPath based on the latest RFC. and then using that JsonPath for translation response(find and udpate) of our api gateway, and other usages, for i18n.
2. using fastparse to implement graphql TQL.

What don't you like, either about the projects you use or the projects you don't? (confusing APIs, docs, missing functionality, etc.)

1. Mill is fast but lack IDEA support, and It would be great to have an interactive shell.
2. Fastparse has great api and document, it's easy to use. when handle jsonpath trailing spaces, I have to write something like identifier ~~whiteSpaces ~/ segments to capture the whitespaces. it would be nice to have something noWhiteSpacesBetween from https://github.com/jsonpath-standard/jsonpath-compliance-test-suite/blob/main/tests/whitespace/selectors.json

{
      "name": "space between root and bracket",
      "selector" : "$ ['a']",
      "document" : {"a": "ab"},
      "result": [ "ab" ]
    },

[Krever]

What company do you work at (if any)?
SwissBorg

How many engineers in the org? (approx is fine)
50-100

Which com-lihaoyi projects?
sourcecode, those included in Scala Toolkit (uPickle, os-lib)

What you like about the projects you use?
Easy-to-use, conciseness

What don't you like, either about the projects you use or the projects you don't? (confusing APIs, docs, missing functionality, etc.)
Not a problem perse but more general remark: we never considered using com-lihayoi ecosystem for production usage. We are focused a lot on the safety of our core system, and those libs probably would not play well with the rest of the stack as well as mentality. Exceptions and lack of effect-tracking-through-signatures are probably the hardest blockers.

Having said that, we are more than happy with using li-hayoi libs for any kind of scripting and automation.

Feel free to message me anywhere (X, Discord, LinkedIn, email) if you need more details.

[coreyoconnor]

What company do you work at (if any)?

https://www.protenus.com

How many engineers in the org? (approx is fine)

~ 30

Which com-lihaoyi projects?

Ammonite is used to provide the shell environment for our internal tools.

os-lib, requests, upickle.

Both are used primarily in operational support tooling. In particular Ammonite.

What you like about the projects you use?

Ammonite is easy to extend and embed. Which is probably why Ammonite appears to be a critical component of various spark shells: almond ; spark connect shell ; one other I forgot.

The other libraries are also appreciated for their ease of use. The stuff that should be easy is easy. Approachable.

What don't you like, either about the projects you use or the projects you don't? (confusing APIs, docs, missing functionality, etc.)

Ammonite has the challenge of requiring a precise set of version requirements. This seems to be inherit in integrating with the scala compiler. Seems like the scala compiler should be providing more functionality here to make this less fragile.

[coreyoconnor]

[beyondpie]

What company do you work at (if any)?
I am a postdoc on Bioinformatics in UCSD.
How many engineers in the org? (approx is fine)
2-3
Which com-lihaoyi projects?
os-lib
What you like about the projects you use?
Exactly the one I need for IO and shell pipeline.
What don't you like, either about the projects you use or the projects you don't? (confusing APIs, docs, missing functionality, etc.)
I hope ammonite could be merged into Scala REPL. I think Scala REPL is easy to use (I use Emacs + Eglot + Metals + SBT-mode), but there are some minor issues (like cannot stop printing long list, function parameters have to be in one line and so on). On the other hand, I cannot open ammonite in my SBT-mode directly like Scala REPL.

[jlvertol]

What company do you work at (if any)?

Glean.ai

How many engineers in the org? (approx is fine)

<10

Which com-lihaoyi projects?

Mill - new Scala projects are built with this, and I plan to migrate the old Sbt projects to Mill as well
Requests - experimented with this to help the Python devs have a Scala codebase that feels familiar, but new projects use STTP because it's included in the Scala Toolkit
uPickle - used in new projects (and whenever I need to patch old projects because Circe is misbehaving)
os-lib - used in scripts
ScalaSQL - used in our newest project

What do you like about the projects you use?

I'll give some backstory first:

I read your book and became acquainted with Mill there, I also read your blog post describing why Mill instead of Sbt, and then I thought "maybe I can build the next project with Mill". I have already used Maven, Gradle and Sbt (I hate them all). Mill seemed promising but it still didn't do what I wanted by default; but you promised the code was straightforward, so I downloaded the source and read it, and... that's when I really started going crazy with your libraries. Your code is actually readable, right level of cleverness, and so when something needs to be changed, I can do it! I feel like I own the codebase, instead of the codebase owning me (heh)

Anyway with that backstory out of the way...

All your projects -> your focus on conciseness, readability, clarity, it's great. Your documentation is pretty good, covers the base use cases, and because your source code can actually be read, the source code provides the rest of the documentation. I also like that you include the rationale for every project, this really helps to understand what niche it's trying to cover.

Mill -> does what I want it to do, easy to learn, defaults make sense. Sbt's default extremely-nested Java-style source folders irk me, and Mill does nothing of that. The build file is actual Scala, I can ctrl+click to go to definitions. It's great, and I can teach it, and it works.

Requests -> best for helping Python devs to move over to Scala, and to show them that Scala can be used "as a better Python"

uPickle -> fast, straightforward, actually works, and you patched the biggest drawback I found (the whole Option[T] deserialization giving a list on Json instead of a null or empty field). JSON is a lightweight format, and I want to use it as such, and this library does just that.

os-lib -> can't write scripts without this! Beautiful and simple!

scalaSQL -> the project vision seemed very promising, the source code is clear, tests are well organized. Gives me hope that it'll be a good replacement for both Slick (does too many things) and ScalikeJDBC (does too few things)

What don't you like, either about the projects you use or the projects you don't? (confusing APIs, docs, missing functionality, etc.)

Mill -> IntelliJ support is rough (this is on JetBrains, I think they'll come to it if adoption continues to increase)
Requests -> even though it was almost a straight port from Python, it didn't behave exactly the same (when it came to attachments or POST actions), causing confusion (version was 0.8.2)
uPickle -> versions before 4.0 didn't behave "intuitively" for the most common use case (the Option[T]/null issue)
scalaSQL -> didn't support schemas. Now we want to use some Postgresql functions (Levenshtein similarity), I'll get back to you with an issue or a PR if we can't use them with rawSql

I'll send you an email.

Thanks for all you've done for the Scala community and ecosystem!

[sideeffffect]

new projects use STTP because it's included in the Scala Toolkit

@lihaoyi does com-lihaoyi have it's own Toolkit? I know TypeLevel have their own as well https://github.com/typelevel/toolkit