[dancer-users] Advent 2015, medium-scale Dancer, part 5: REST APIs and JSON

[Warren Young]

In previous parts of this article series, I defined each route handler as a one-liner, like so:

    prefix '/mf' => sub {
        get '/' => sub { retrieve(context); };
        prefix '/subfeature' => sub {
            get  '/'    => sub { _get(context); };
            post '/'    => sub { _post(context); };
            put  '/'    => sub { _put(context); };
            del  '/:id' => sub { _del(context); };
        };
    };

This is very much on purpose, despite the fact that Dancer lets you define all of the code for each route handler within the `sub { ... }` block. Why add the extra function call layer?

Simple: I want to make the URL structure of my web app explicit in the code.

To some extent, you can see this by looking at your `views` directory, since a prior article in this series encouraged you to map view files 1:1 to Dancer routes, as much as possible. The problem with relying on the `views` directory contents as a document for the web app's URL structure is that the OS's file system normally shows you only one layer of the directory hierarchy at a time. You have to go out of your way to see the whole tree in a compact form, on a single screen. That means you may never actually find yourself studying the hierarchy as a whole, and thus may never consider whether it is a cohesive design.

This practice of defining Dancer route handlers as one-liners solves that. It ensures that every time I adjust the route handlers for my site, I am doing so while looking at the neighboring routes; I define new routes in context, not independently. This counteracts the entropic forces that result in API design drift, because you are implicitly encouraged to define new routes that fit logically into the scheme you defined previously. Without this extra Perl function call layer, the route definitions are visually broken up, so that you cannot see the whole design on a single screen. The resulting URL scheme can turn into a mess, as developers hack on it over the course of years.

This is also why I have carefully lined up the blocks and [fat commas](https://en.wikipedia.org/wiki/Fat_comma) in the route definitions: I want patterns in the URL scheme to jump out at me. Left-justifying everything obscures these patterns.

Now you may be asking, why does it matter that the URL scheme be "clean?" Simple: laying out your route handlers like this makes patterns graphically apparent. Just by looking at the code, you may see patterns that you never would have noticed when the code is arranged as in the first part of this article series. Some of those patterns may point out some previously-hidden practical value, in the same way that graphing a numeric data set sometimes does.

It turns out that the example route scheme we've been defining throughout this article series includes a web API; it's just been sitting there, waiting to be discovered. With a bit of polishing, it might even turn into a proper [REST](https://en.wikipedia.org/wiki/Representational_state_transfer) API. Let's try.

Look back up at that route definition block above. What does it remind you of? If you said [CRUD](https://en.wikipedia.org/wiki/Create,_read,_update_and_delete), go to the head of the class.

Now, HTTP verbs do not correspond directly to CRUD, because HTTP was not designed to be a CRUD API. There are competing design philosophies when it comes to mapping HTTP verbs to the CRUD model of persistent data storage.

I prefer a simple 1:1 mapping:

| DB term    | HTTP verb
|------------|----------
| C = create | `POST`
| R = read   | `GET`
| U = udpate | `PUT`
| D = delete | `DELETE`

The R = `GET` and D = `DELETE` rules are obvious, but it may not be clear to you why the other two rules are correct.

[The HTTP spec](http://www.w3.org/Protocols/rfc2616/rfc2616.html) defines `POST` and `PUT` so that they're nearly interchangeable, which is how we ended up with competing REST/CRUD API design philosophies.

Some API designers like to use `POST` for both create and update operations, since you can distinguish the cases based on whether the HTTP request contains a database ID: if so, it's an update for an existing record, else it's a create operation.

I don't like doing this in Dancer code because it means you need to define two different operations in a single route handler. This design choice makes more sense in a web framework that defines the URL scheme in the file system, such as PHP, ASP, or JSP, since it is normal to handle all four verbs in a single file in that sort of web framework.

Although the HTTP spec defines `PUT` and `POST` as partially interchangeable, you must not swap the `PUT` and `POST` verbs from the way I've defined them above. The HTTP spec says that [`PUT` is idempotent](http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html#sec9.1), meaning that the browser is allowed to cache `PUT` requests, suppressing subsequent identical calls if the request parameters do not change. In CRUD terms, this means you cannot create two records with identical contents, but different IDs, which means you cannot define create as `PUT`. Browsers are not allowed to treat identical `POST` calls as idempotent, so we use that verb for CRUD's create operation, leaving `PUT` = update.

That's enough design philosophy. How does all that affect our Dancer code? We might decide that since we've almost got a REST API here, that we might want to push it the rest of the way. We might move some of the code from `lib/App/MajorFeature.pm` to `lib/App/API/SubFeature.pm`:

    prefix '/api' => sub {
        prefix '/subfeature' => sub {
            post '/'    => sub { _post(context); };
            get  '/'    => sub { _get(context); };
            put  '/:id' => sub { _put(context); };
            del  '/:id' => sub { _del(context); };
        };
    };

My choice to use leading underscores in these module-internal route handler function names lets us parallel the Dancer DSL keyword naming scheme.

You might instead prefer to make the HTTP-to-CRUD mapping more explicit:

    prefix '/api' => sub {
        prefix '/subfeature' => sub {
            post '/'    => sub { _create(context); };
            get  '/'    => sub { _read(context); };
            put  '/:id' => sub { _update(context); };
            del  '/:id' => sub { _delete(context); };
        };
    };

As in the previous article in this series, the leading underscore convention saves us some aggravation here, since `delete` is a Perl keyword, and `read` is a core Perl function.

The only other thing to change here is that REST APIs normally return JSON or XML. I prefer JSON, since that's directly usable by the JavaScript code that made the Ajax call to one of these APIs. One of the best features of Dancer is that automatic JSON encoding is built right in. We might define the `_create()` function referenced above as:

    sub _create {
        my ($ctx) = @_;		# see Part 2 for the definition of context()
        
        my $params = $ctx->{request}->params;
        my $db     = $ctx->{dbconn};
        
        if ($db->create_something($params->{arg1}, $params->{arg2})) {
            return {
                success => JSON::true,
                id      => $db->last_insert_id(),
            };
        }
        else {
            return {
                success => JSON::false,
                error   => 'failed to create record: ' . $db->last_error(),
            };
        }
    }

In Dancer, a hash reference returned from a route handler is automatically JSON-encoded, and the HTTP response's content type is set to `application/json`. Parsing and using that reply on the client side in JavaScript is therefore trivial.

If your new REST API route handlers previously returned HTML, and you have existing code that still needs this data in HTML form, I prefer to separate those Ajax call handlers into a different section of the URL hierarchy: `/parts`. The rule is simple: `/api` routes return JSON, `/parts` routes return HTML fragments. Those fragments are intended to be inserted into an existing DOM via JavaScript. The `/parts` section of the URL hierarchy separates out such route handlers from those that return whole HTML pages.

This scheme gives you a utility function that returns a hashref to other parts of your Dancer app so they can build HTML from it, but returns JSON when called from a Dancer route handler.


**Conclusion**

Well, that's it, the end of this article series. We have gone from a monolithic app structure as generated by `dancer2 -a App` to a nicely-factored application that's ready to grow into the thousands of lines of code. Along the way, we discovered that we had a REST API sitting there in our app's URL structure all along, and matched up the rest of the URLs to resources in a way that will make debugging simpler by following some sensible naming conventions.

I hope this reduces some of the growing pains in your own Dancer applications!