Building a small microservice in Haskell

Posted on November 1, 2019 by wjwh

Some days ago there was an article posted on Hacker News about Haskell and the usual debate started regarding the practicality of Haskell in “real world” applications. In particular, there were several comments like this one that bemoaned the lack of intermediate level tutorials on how to write functioning Haskell programs.

Knowing Haskell fairly well and having a day job that basically consists of making microservices in Ruby, I decided to make a very basic “curl-only” url shortening microservice to see how far I’d get. It does not contain any frontend code, but adding that would be simple enough. I find that URL shorteners are small enough to allow focusing on the language instead of on the business logic. It’s a good exercise when learning a new backend language, similar to how every JS framework tutorial seems to start with a TODO app.

I’ll assume you already have proficiency in the HTTP request/response model and basic Haskell syntax. I also will not cover deployment with docker or similar or production-level bulletproofing like rate limiting etc.

The App

We’ll build a simple url shortening service that will:

Have a /healthcheck endpoint because every app should have one. It always returns 200.
Have an endpoint on /new-url that you can POST to with an URL that you want shortened. It’ll return the shortened url.
Allow GET-ing such shortened urls and respond with a redirect to the longer url or to a default page if the page could not be found.

As a backend store we’ll use Redis, because I like it a lot and we’ll get expiry of urls for free. Urls will stay valid for a week and then they get deleted. We’ll use the Scotty library for making our webapp. If you know frameworks like Sinatra or Kemal, you’ll immediately feel at home.

Setting up

Assuming you already have stack installed, to generate a basic app skeleton run stack new shorturls scotty-hspec-wai in the terminal. To test if everything went well, cd shorturls then stack test to run the autogenerated test suite. If you have not installed many Haskell libraries before this will take a fair amount of time as it installs everything including the compiler, but subsequent invocation of stack test will be faster. You can also stack run to get a web server running on port 8080.

Regarding the file structure, /app contains code that is only required for running the program and /test contains code only used in testing. They can both import code from /src, which is probably where most of your code should live. shorturls.cabal contains a lot of useful information, like information about the version, author, compile and runtime flags and more. Most interesting for now are the build-depends sections, which list the libraries that the app depends on. This is similar to a Gemfile in Ruby, requirements.txt in Python or package.json in JS. It already contains most of the code we want for the web serving part, but we also want to access Redis. There is an excellent library called hedis which we can use for this, so add two extra line at the end of the build-depends section of the library section for hedis and random (don’t forget the commas!). Running stack test or stack run again will install hedis and random before proceeding.

As mentioned, most code should live in /src. There is already a file called Example.hs with a pregenerated Scotty app in it. All the example code in app' can be deleted. We’ll need to import the libraries we need to generate random strings and to work with Redis. This gives a small problem: both Scotty and Hedis will define a get function, so one of these libraries will need to be imported qualified.

A few endpoints

This section is much better understood if you have the file open.

Most of the interesting parts live in the app' function in /src/Example.hs, which defines the endpoints and what to do when they are called. It takes a Hedis.Connection object, which is actually more like a pool than a single connection. It also accepts a lazy Text value defining a default URL to redirect to if the one requested could not be found.

The healthcheck is fairly straightforward: we only need to return a 200 response, the body does not really matter. All that is needed in the app' function is the following line:

  get "/healthcheck" $ text "I'm OK. Thanks for asking!"

This defines an handler for GET requests to /healthcheck which returns a Text value. Now we can run stack run again and verify it works:

$ curl localhost:8080/healthcheck
I'm OK. Thanks for asking!

Making a new shortened URL is a little bit more interesting, since it involves IO actions to generate the random string and to store it in Redis. Since app' is in the ScottyM monad, the IO actions need to be lifted before they can be used:

  post "/new-url" $ do
    longURL <- param "long_url"
    randomID <- liftIO newRandomID
    storeUrl connpool randomID (TE.encodeUtf8 longURL)
    text $ "http://my-domain.org/" <> (TL.fromStrict . TE.decodeUtf8 $ randomID)

This does basically what you’d expect: read the URL to be shortened from the POST parameters, generate a new random string for the shorturls, then put it in Redis. Finally, we concatenate the random ID to our URL and return the shortened URL. There is a lot of converting between ByteString and Text in this going on, because Hedis expects everything to be strict ByteStrings, while Scotty wants lazy Text values everywhere. Performing a GET request against the returned url would be handled with the last endpoint:

  get "/:random_id" $ do
    randomID <- param "random_id"
    maybe_long_url <- liftIO $ Hedis.runRedis connpool (Hedis.get randomID)
    case maybe_long_url of
      Left _ -> redirect defaultUrl -- error returned by redis
      Right Nothing -> redirect defaultUrl -- redis call succeeded but the key was not present
      Right (Just long_url) -> redirect . TL.fromStrict . TE.decodeUtf8 $ long_url

Similar to the /healthcheck endpoint above, this will respond to GET requests, but unlike the previous examples, this endpoint will ‘capture’ the part after / into the random_id parameter which can be accessed in the request handler. This is (intentionally) very similar to how Sinatra works in Ruby. After retrieving the parameter, we look in Redis if a key exists or not. Based on the result of that lookup, we either redirect the user to the stored long URL or to a default URL. Note that unlike most languages, we explicitly have to handle the case where (connecting to) Redis returned an error.

Testing

The test files are in the test folder. The stack template will only autogenerate a single file for us to begin with, but it’s all we need so that is fine. The tests are basically what you’d expect if you spent some time writing HTTP microservices: you set up the database as required, hit the endpoint with some predetermined parameters and verify that the response conforms to what you expect.

Compared to dynamic languages, writing tests for Haskell is definitely different. The type system greatly reduces the need for most ‘boilerplate unit tests’, but on the other hand it is MUCH harder to (for example) monkeypatch the random string generator to return exactly what you want or to force Redis to return an error.

As a nice bonus, stack comes with a test coverage generator built in that will output nicely colored HTML pages of your code indicating which code paths are never checked. You can access it with stack test --coverage.

Conclusion

This is not a complete service. For example, it does not validate whether a submitted string is actually a valid URL, it has no way to delete or alter an existing URL and it’s even possible that new urls overwrite already existing ones. We could also do a lot more to capture the domain in types. I hear the Servant library makes it easier to ensure only properly typed requests get through, but I was already vaguely familiar with Scotty so I did not look further into Servant.

I spent way more time fighting the type system than I expected. I already mentioned having to convert between strict ByteString and lazy Text earlier. Haskell has no less than five (!) commonly used types for representing strings. Some of the non-WAI expectations is the tests also had to be liftIO’d to typecheck properly, which took me way too long to figure out. On the other hand, having instant re-typechecking with ghcid makes the whole process pretty nice. It’s definitely faster and better than rerunning tests in Ruby or JS.

Overall, I’d rate the experience as “fine”. Defining the endpoints and writing tests for them was pretty much on the same level of ease as writing a similar service in Ruby with Sinatra, maybe a little bit more difficult because of the monad lifting and manual Text/ByteString conversions. On the other hand, you don’t need to test as much because the type system will automatically prevent you from calling methods on nil. It runs a lot faster than dynamic languages just by virtue of being compiled and has proper multithreading built in, but you could get that just as easily from Go, Crystal or Rust. One thing that Haskells type system does provide but that won’t properly kick in until the code gets much larger, is that the strictness of hte type system makes refactoring a breeze. I don’t think I’ll start using Haskell for ‘everything’, but the next time I encounter something that plays to its strengths it definitely has a chance. I hope you’ll give it a try, too!

The complete code can be found here.