# REST In Depth Learn about more advanced [REST][rest] concepts like [HATEOAS][hateoas], and the various ways REST APIs are implemented in the wild, from "practical REST" to "true REST". **Recommended reading** - [REST introduction & HTTP](../rest/) --- class: center, middle, image-header ## Richardson Maturity Model .breadcrumbs[
REST In Depth
]
--- ### How RESTful is my API? .breadcrumbs[
REST In Depth
>
Richardson Maturity Model
] .grid-40[ The technologies of the World Wide Web form an amazing **technology stack** you can pick from when developing your REST API. However, not everyone understands all of these technologies or how to use them to their fullest potential. ] .grid-60[
] .container[ The [REST maturity model developed by Leonard Richardson][rest-maturity] splits the web technologies used to develop REST APIs into a stack with 4 levels. ] --- ### REST maturity level .breadcrumbs[
REST In Depth
>
Richardson Maturity Model
] Web developers tend to pick technologies from the bottom of the stack:
If you use none of these technologies, you are at level 0 and your API cannot be called REST. The more web technologies you use, the greater your "REST maturity level". --- ### "Not really REST" APIs - Levels 0 & 1 .breadcrumbs[
REST In Depth
>
Richardson Maturity Model
] .grid-50[ **Level 0** One URI, one HTTP method. For example: XML-RPC and most SOAP services. This looks more like a library of functions than a web service. It's a huge **black box** with a single web-based peephole. Definitely not REST: the whole service consists of one overly complex resource. ] .grid-50[ **Level 1** Many URIs, one HTTP method. The web service has been **split into separate components** that each get their own URI, which makes it much easier to understand. It's a bit more RESTful, but not really there yet. This web service deals more with **actions rather than resources**. Real-world example: [Flickr API][flickr-api]. ] --- ### "Practical REST" APIs - Level 2 .breadcrumbs[
REST In Depth
>
Richardson Maturity Model
] HTTP introduces [standardized methods][http-methods] (or verbs): | Method | Description | [Safe][http-safe] | [Idempotent][http-idempotence] | | :------- | :------------------------------------------- | :---------------: | :----------------------------: | | `GET` | Retrieve a resource at an URL. | ✔ | ✔ | | `POST` | Create a new resource subordinate to an URL. | ✘ | ✘ | | `PUT` | Replace (modify) the resource at an URL. | ✘ | ✔ | | `PATCH` | Partially modify a resource at an URL. | ✘ | ✘ | | `DELETE` | Delete the resource identified by an URL. | ✘ | ✔ | You do not have to reinvent the wheel for the most common operations on resources: creating, reading, updating and deleting ([CRUD][crud]). Each method also comes with **guarantees** that are the same for every API in the world: [`GET`][http-get] is [safe][http-safe] (it changes nothing) and [idempotent][http-idempotence] (repeating it changes nothing). **Where there are constraints there are optimizations**: [caching][http-cache], [conditional requests][http-conditional], [partial requests][http-partial] etc. Real-world examples: [AWS Simple Storage Service (S3) API][s3-api], [Stripe API][stripe-api]. --- ### "True REST" hypermedia APIs - Level 3 .breadcrumbs[
REST In Depth
>
Richardson Maturity Model
] An API is not fully REST in the strictest sense if it does not use [**Hypermedia as the Engine of Application State (HATEOAS)**][hateoas]. This is what distinguishes REST from other architectures and what makes a "true REST" API. The basic principle of HATEOAS is that **the client interacts with the server dynamically through [hypermedia][hypermedia]**. Since the World Wide Web exhibits properties of the REST architecture, let's take a look at HTML: ```html
My Awesome Website
Welcome, visitor!
Learn more about me
Buy some stuff
</body> ``` HTML is hypermedia. **Why?** Think about the following question: what drives the user's interaction through the various pages of a website? How do they navigate? --- #### Hypermedia .breadcrumbs[
REST In Depth
>
Richardson Maturity Model
>
"True REST" hypermedia APIs - Level 3
] An HTML page provided by a server, such as a web page, contains 2 kinds of hypermedia: [hyperlinks][hyperlink] and [forms][html-form]: - **Hyperlinks** tell the client **where other resources are**. - **Forms** tell the client **what it can do to the resources**. To use a website, the client (i.e. the browser) only needs to: - Know the entry point (e.g. `http://example.com`). - Understand the HTML format (the `text/html` [media type][media-type]): not only how to display it, but also that it contains `
` tags that can be followed, and `
` tags that describe how to modify data. **The client does not need to (and should not) know the entire structure of your website** (i.e. all your URLs) in advance. When a server sends you HTML, it is not only sending you data to display, it is also sending you hypermedia: the relations between the current resource and other resources. Those **relations are as important as the data itself** to the functioning of the website. --- #### Hyperlinks in HTTP .breadcrumbs[
REST In Depth
>
Richardson Maturity Model
>
"True REST" hypermedia APIs - Level 3
] [RFC 8288 (Web Linking)][web-linking] defines the `Link` HTTP header which can be included in non-HTML responses to provide links to other resources: ```http HTTP/1.1 200 OK Content-Type: application/json *Link:
; rel="prev"; title="previous chapter", *
; rel="next"; title="next chapter" { "title": "Once Upon A Time", "contents": "Lorem ipsum..." } ``` Using this header, the server can tell the client where to find related resources, without the client having to know how to build these URLs. --- #### Format of the `Link` header .breadcrumbs[
REST In Depth
>
Richardson Maturity Model
>
"True REST" hypermedia APIs - Level 3
] Multiple links in the header are comma-separated. Each link looks like this: ```
; `rel="first prev"` ``` It contains: - The **target URL** between `<>` - One or more **parameters** preceded by `;`: The `rel` (or "relation") parameter is mandatory, as it indicates **what kind of link it is**. There is a [registry of official relation types][iana-link-relations] (such as `first`, `prev`, `next` and `last` for collections). You can use your own custom relation types but instead of single words, they should be URIs: ```
; `rel="http://example.com/my-rels/rtfm"` ``` You don't have to build this format by hand. You can probably find a package that does it for you. For example, a quick [search for "link header"][npm-search-link-header] in the npm registry suggests several packages: [format-link-header][npm-format-link-header], [parse-link-header][npm-parse-link-header], [http-link-header][npm-http-link-header], and more. --- #### The OPTIONS method in HTTP .breadcrumbs[
REST In Depth
>
Richardson Maturity Model
>
"True REST" hypermedia APIs - Level 3
] The [`OPTIONS` method][http-options] is another method like `GET` or `POST`. It can be used by the client to ask the server what are the communication options of a given resource: ```http OPTIONS /api/people/123 HTTP/1.1 ``` This could be a response to that request: ```http HTTP/1.1 204 No Content Allow: OPTIONS, GET, HEAD, PUT ``` Using the [`Allow` header][http-allow], the server is telling the client: you can also use the methods `GET`, `HEAD` and `PUT` on this resource. Since those methods have a standard meaning the client can deduce, for example, that this resource **can be modified** (because you can use the `PUT` method). > This is an example of the server telling the client how a resource can be > modified, similarly to forms in HTML. --- #### Hyperlinks in REST APIs .breadcrumbs[
REST In Depth
>
Richardson Maturity Model
>
"True REST" hypermedia APIs - Level 3
] There are hypermedia formats similar to HTML but meant for APIs: - [JSON:API][json-api] - [Collection+JSON][collection-json] - [Hypertext Application Language (HAL)][hal] - [(Draft RFC)][hal-rfc] - [JSON for Linking Data (JSON-LD)][json-ld] - [(W3C Spec)][json-ld-spec] - [JSON Hyper-Schema (RFC Draft)][json-hyper-schema] - [Siren][siren] For example, HAL+JSON defines a standard way of representing hyperlinks in JSON (it also uses the standard IANA link relations): ```json { * "_links": { * "self": { * "href": "http://example.com/api/book/hal-cookbook" * } * }, "id": "hal-cookbook", "name": "HAL Cookbook" } ``` --- #### Custom Hypermedia Type .breadcrumbs[
REST In Depth
>
Richardson Maturity Model
>
"True REST" hypermedia APIs - Level 3
] You may also define your own custom media type, also known as a [vendor-specific media type][vnd-media-type]. For example, instead of using raw JSON, you can create a `application/vnd.my-awesome-website+json` media type. The `+json` suffix indicates that your media type is an augmentation of the basic JSON media type with additional rules. It's up to you to define what these rules are. For example, you could write the following in your documentation: > All JSON properties ending with `_link` represent a hyperlink to a related > resource: ```json { "first_name": "John", "last_name": "Doe", * "address_link": "https://example.com/api/people/jdoe/address" } ``` You could even [officially register your media type at IANA][iana-media-types]. --- #### To HATEOAS or not to HATEOAS? .breadcrumbs[
REST In Depth
>
Richardson Maturity Model
>
"True REST" hypermedia APIs - Level 3
] .grid-50[ **Advantages:** Clients can follow link relations in your API instead of hardcoding URLs. This **reduces the coupling between client and server**. The server may refactor the URL structure at will, and influence the client by including/omitting links. Using a standardized hypermedia format **makes your API browsable** by both humans and automated tools (e.g. [HAL browser][hal-browser]). If the format is widely used, existing clients may even be able to manipulate your data out of the box. ] .grid-50[ **Disadvantages:** There is no free lunch. It requires **more work** to implement a level 3 REST API. **Clients must follow the hyperlinks** between resources instead of accessing hardcoded URLs directly, or all the benefits of hypermedia are lost. There is **no clear standard** in the industry when it comes to choosing your hypermedia format. Hypermedia is not a silver bullet: clients must still know your domain. ] .container[ Real-world examples: [GitHub v3 API (custom media type)][github-api], [Foxy REST API (HAL+JSON)][foxy-api]. ] --- ## URL structure .breadcrumbs[
REST In Depth
] Resources often have relationships: - A resource which represents a **collection** has items (e.g. users/user). - Blog **posts** have **comments**. - **Courses** are taught by **professors**. How should we define the URLs to retrieve those resources? --- ### Retrieving a collection or a single resource .breadcrumbs[
REST In Depth
>
URL structure
] Assume there is a resource which represents the list of users in a blog: ``` http://blog.example.com/users ``` What should be the URL that identifies user number 2? ``` http://blog.example.com`/user/2` http://blog.example.com`/users/2` ``` --- #### What should the URL of a single resource be? .breadcrumbs[
REST In Depth
>
URL structure
>
Retrieving a collection or a single resource
] Technically, the second solution is more correct with regards to URL structure and HTTP: ``` http://blog.example.com/users (collection) http://blog.example.com/users/2 (item in the collection) ``` - `/users/2` expresses that user number `2` is somehow contained within the `users` collection. The URL expresses the **hierarchical relationship** between the two resources. - When creating a user with a `POST /users` request, the HTTP specification states that the created resource should be a **subordinate** of the resource identified by the request, so it makes sense to reflect that relationship in the URL. - Many prefer to use `users` in both the collection and item URL so you do not have to worry about switching between singular and plural (e.g. `people/person`). > This is a matter of debate. You will see both solutions implemented in > real-world APIs. What's important is not to mix the two techniques to avoid > confusion. --- ### Nested (or hierarchical) URLs .breadcrumbs[
REST In Depth
>
URL structure
] Suppose I want to get the comments on post 892 of a blog. The URL can be **nested** and mirror the **hierarchical structure** between the resources: ``` http://blog.example.com`/posts`/892`/comments` ``` Or it can be **flat** (only one path level) and have _query parameters_ like this: ``` http://blog.example.com`/comments`?postId=892 ``` In this case, the **nested** or **hierarchical** version probably makes more sense if you assume that: - A post comment **cannot exist** outside of a post - You'll probably **never want to display all comments** across posts, only the comments of a specific post --- ### Flat URLs .breadcrumbs[
REST In Depth
>
URL structure
] Suppose I want to retrieve the courses given by professor Arnold. Again, the URL could be **nested**: ``` http://heig-vd.ch`/professors`/arnold`/courses` ``` Or it could be **flat**: ``` http://heig-vd.ch`/courses`?professor=arnold ``` In this case, the **flat** version probably makes more sense because: - The existence of the course **does not depend** on the existence of the professor (if Arnold retires, someone else will take over the course) - You'll probably want to be able to **list all courses** on a page somewhere --- ### Nested vs. flat URLs .breadcrumbs[
REST In Depth
>
URL structure
] There is no _right or wrong_ answer. You will find both styles in popular REST APIs. When defining your API's URLs, you should ask yourself: > Is there an **ownership** or **composition** relationship between these resources? In this case, it makes sense to use **nested** or hierarchical URLs. > Will I need to **display all the resources** of this type on a page somewhere? In that case, it makes sense to use **flat** URLs. --- ### Using both nested & flat URLs .breadcrumbs[
REST In Depth
>
URL structure
] It's also possible to define **multiple collections** for the same resource at **different paths**: ``` GET /courses GET /professors/arnold/courses ``` - The first collection (`/courses`) can be used to list **all courses** - The second collection (`/professors/arnold/courses`) can be used to list **the courses taught by professor arnold** These two collections both produce a list of resources of the same type (courses), but **they are different collections**. Their contents will vary over time, and most of the time the two collections will not produce the same result (there are courses taught by other professors). --- ### Filtering with URLs .breadcrumbs[
REST In Depth
>
URL structure
] Which URL structure is better for filtering parts of a collection? ``` GET /employees/byDepartment/accounting GET /employees/byLastName/Smith GET /employees?department=accounting GET /employees?lastName=Smith ``` The first two are simply not practical. What happens when you want to retrieve all people named Smith in the accounting department? Either you cannot, or you will end up with clunky, unintuitive URLs: ``` GET /employees/byDepartmentAndLastName/accounting/Smith GET /employees/byLastName/Smith/andByDepartment/accounting ``` --- #### Use URL query parameters for filtering .breadcrumbs[
REST In Depth
>
URL structure
>
Filtering with URLs
] The path of the URL is not meant to contain multiple dynamic parameters. However, the query string exists for just this purpose: ``` GET /employees`?`department=accounting GET /employees`?`department=accounting`&`lastName=Smith GET /employees`?`lastName=Doe`&`department=sales ``` You can combine query parameters however you want (provided the implementation of the server supports it). It is also more logical to have the single resource `GET /employees` which you can filter with additional parameters. In the end, all responses to these requests are lists of employees. Always prefer query parameters to filter collections. --- ## Resources vs. actions .breadcrumbs[
REST In Depth
] Fundamental to the REST architectural style is the idea that you are performing actions on resources. In the context of the web this usually means: - The **HTTP method** is the **action**: `GET`, `POST`, `PUT`, etc. - The **URL** identifies the **resource**, i.e. the thing being manipulated. Which ones of these HTTP requests do you think are RESTful? - `GET /api/comments` - `POST /api/users` - `PUT /api/users/:id` - `POST /api/users/:id/enable` - `POST /api/users/:id/disable` - `POST /api/users/:id/follow` - `POST /api/comments/:id/star` - `POST /api/comments/:id/unstar` --- ### Modeling actions with REST .breadcrumbs[
REST In Depth
>
Resources vs. actions
] The last 5 requests are technically **not RESTful**, for example: - `POST /api/users/:id/enable` The URL ends with **a verb instead of a noun**, therefore: - The URL represents an action (verb) instead of a resource (noun). - There are 2 actions: the HTTP method, `POST`, and the one in the URL, `enable`. In theory, the only action should be the HTTP method, and the URL should always represent the "thing" being acted upon, not an action. --- ### REST actions as properties .breadcrumbs[
REST In Depth
>
Resources vs. actions
] Take the following non-RESTful actions: - `POST /api/users/:id/enable` - `POST /api/users/:id/disable` They could be modeled as a user property instead: ```json { "username": "jdoe", * "enabled": true } ``` You could then update that property with `PUT` or `PATCH`. The following call could be made to disable the user: ```http PATCH /api/users/jdoe HTTP/1.1 Content-Type: application/json { "enabled": false } ``` --- ### REST actions that should really be resources .breadcrumbs[
REST In Depth
>
Resources vs. actions
] Take the following non-RESTful action: - `POST /api/users/:id/follow` It's very likely that at some point you'll be needing the list of a user's followers. This should really be a collection resource that you can add followers to: - `POST /api/users/:id/followers` That way, you can simply implement this call in the future to list a user's followers: - `GET /api/users/:id/followers` --- ### REST actions as a sub-resource .breadcrumbs[
REST In Depth
>
Resources vs. actions
] Take the following non-RESTful action: - `POST /api/comments/:id/star` - `POST /api/comments/:id/unstar` If you assume that you can only only star/unstar a comment (i.e. there is only one star, not a range from 1 to 5), you can model it as a single sub-resource of each comment: - `PUT /api/comments/:id/star` - `DELETE /api/comments/:id/star` > The first version uses the **verb** "star", while the second version uses the > **noun** "star", with the HTTP method indicating the action. Also note that > "star" is used in the singular, since there is only one. --- ### REST actions as a collection .breadcrumbs[
REST In Depth
>
Resources vs. actions
] Take the following non-RESTful actions: - `POST /api/users/:id/enable` - `POST /api/users/:id/disable` We already described one way of modeling these as a property. However in some cases, you might need to remember all actions that were taken on a resource for **auditing** or **security** purposes. For example: GitHub has a page that lists all security-related actions on your user account, so that you might see if someone else obtained access to your account and did something bad. In that case, you might want to have a collection resource that represents the actions taken on a given user: - `POST /api/users/:id/actions` - `GET /api/users/:id/actions` --- #### Using a collection of actions .breadcrumbs[
REST In Depth
>
Resources vs. actions
>
REST actions as a collection
] With such a collection, you would `POST` to create actions (the `type` property determines what action to take): ```http POST /api/users/jdoe/actions HTTP/1.1 Content-Type: application/json *{ "type": "disable", "reason": "He looks suspicious." } ``` ```http HTTP/1.1 201 Created Location: https://example.com/api/users/jdoe/actions/321 ``` Then later you could `GET /api/users/jdoe/actions` to retrieve the list of actions that were performed: ```http HTTP/1.1 200 OK Content-Type: application/json [ { "type": "disable", "reason": "He looks suspicious.", "date": "2010-01-01" }, { "type": "changePassword", "date": "2008-03-01" }, { "type": "confirmEmail", "date": "2008-01-01" } ] ``` Now you have all the data you need for that audit/security page. --- ### So what do nouns buy you? .breadcrumbs[
REST In Depth
>
Resources vs. actions
] Respecting the rule is not the point. The point is **who else can understand your request**. .grid-50[ **Verbs in the URL** ```http POST /api/comments/42/star POST /api/comments/42/unstar ``` Two endpoints, and **two more for every new capability**. `star` and `unstar` are words that only _your_ application understands. Is it safe to send one of them twice? Only you know: `POST` is neither safe nor idempotent. ] .grid-50[ **A noun in the URL** ```http PUT /api/comments/42/star DELETE /api/comments/42/star ``` One resource. `PUT` and `DELETE` already mean "make it so" and "remove it", so there is no need to invent `unstar`. Both are [idempotent][http-idempotence] _by definition_: send either ten times, the result is the same. ] .container[ Modeling with resources moves the meaning of a request **out of your vocabulary and into the protocol**, where code that knows nothing about your application can still do the right thing: cache the response, retry the request, or refuse it if the resource has changed in the meantime. ] --- class: compact-table #### The benefit you get immediately .breadcrumbs[
REST In Depth
>
Resources vs. actions
>
So what do nouns buy you?
] Caching and retries mostly benefit **other people's code**. There is one benefit you get right away, on your own project: naming a resource forces you to find **concepts you were about to lose**. | The action you wanted | The thing it really was | What it got you | | :------------------------ | :--------------------------------------------------- | :-------------------------------------------------------- | | `POST /users/:id/follow` | a follower in a collection | `GET /users/:id/followers`, the list you need anyway | | `POST /comments/:id/star` | a `star` sub-resource | `DELETE` unstars it: one concept, not two endpoints | | `POST /users/:id/disable` | an `enabled` property, or an entry in an audit trail | a state clients can **read**, or `GET /users/:id/actions` | In each case, asking "what is the **thing**?" instead of "what is the **action**?" uncovered something the application needed anyway. > On a small project with a single client, an "action" API would work too. These > benefits grow with the number of clients, the number of endpoints and the > lifetime of the API. That is why REST is an **architectural** style: it is > designed for systems that must survive their first version. --- ## Linked resources .breadcrumbs[
REST In Depth
] This section will discuss different ways to represent links between resources as well as their pros and cons. Suppose that your RESTful API handles **movies** and **directors** where one movie has **one** director, and that it provides the following operation: ``` GET /movies/:id ``` You will retrieve the movie's data, of course, but what do you expect to see concerning the movie's **director**? --- ### Embedded resource .breadcrumbs[
REST In Depth
>
Linked resources
] The director can be **embedded** into the movie's data: ```json { "id": "oa9ufne", "title": "The Fellowship of the Ring", "rating": 8.9, * "director": { * "id": "la09sld", * "name": "Peter Jackson", * "gender": "male" * } } ``` - _Advantage:_ it **reduces chattiness**: fewer requests have to be made to the server to retrieve both the movie's and the director's data. - _Disadvantage:_ **more data** is exchanged between client and server **that the client might not always need**. --- ### Resource reference via ID .breadcrumbs[
REST In Depth
>
Linked resources
] To save bandwidth, only a unique identifier for the director could be included: ```json { "title": "The Fellowship of the Ring", "rating": 8.9, * "directorId": "la09sld" } ``` - _Advantages:_ - **Smaller JSON payloads**. - _Disadvantages:_ - It **increases chattiness**: the client has to make two requests to retrieve both the movie's and the director's data. - The client **must know how to build the URL** to the director resource, which is not always obvious (e.g. you could think it's `/directors/la09sld` but it could be `/people/la09sld`). > This would be considered to be at REST maturity level 2 because it is not > hypermedia. --- ### Resource reference via hyperlink .breadcrumbs[
REST In Depth
>
Linked resources
] Instead of just the ID, you could include a hyperlink to the director's resource: ```json { * "_links": { * "http://example.com/links/director": { * "href": "http://example.com/api/people/la09sld" * } * }, "title": "The Two Towers", "rating": 7.1 } ``` - _Advantage:_ **decouples** the client from your API through **hypermedia**: the client can browse the API without knowing your URLs in advance ([HATEOAS][hateoas]). - _Disadvantages:_ it **increases chattiness**. It requires **more requests by the client** to follow link relations instead of accessing URLs directly. > This would be REST maturity level 3. If you use this solution, you should use an appropriate media type instead of > plain JSON. The above example uses [HAL][hal] but there are others like [JSON:API][json-api], or you can > define your own. --- ### Optional resource embedding .breadcrumbs[
REST In Depth
>
Linked resources
] Again, there's no _right or wrong_ answer: it depends on your use case. By spending a little more time on your server's implementation, it's also possible to **combine** some of these patterns: .grid-50[ ```http GET /api/movies/312 ``` ```http HTTP/1.1 200 OK Content-Type: application/json { "title": "The Two Towers", "rating": 7.1, "directorId": "la09sld" } ``` ] .grid-50[ ```http GET /api/movies/312?`include=director` ``` ```http HTTP/1.1 200 OK Content-Type: application/json { "title": "The Two Towers", "rating": 7.1, "directorId": "la09sld", * "director": { * "id": "la09sld", * "name": "Peter Jackson", * "gender": "male" * } } ``` ] .container[ This **lets the client decide** what's most appropriate **for each request** (depending on the view it is rendering). ] --- ### Embedding a collection .breadcrumbs[
REST In Depth
>
Linked resources
] You can ask yourself the same questions for **one-to-many** or **many-to-many** relationships: .grid-55[ ```json { "name": "John Doe", "addresses": [ { "street": "8785 Windfall St.", "city": "Whitehall, PA 18052" }, { "street": "1 N. Cactus Ave.", "city": "Green Bay, WI 54302" } ] } ``` ] .grid-45[ ```json { "name": "Apple", "employees": [ { "firstName": "Tim", "lastName": "Cook", "title": "CEO" }, { "firstName": "Jony", "lastName": "Ive", "title": "CDO" } ] } ``` ] .container[ When embedding collections, be mindful of the **amount of data**. Embedding a person's addresses is probably reasonable, as few people will have many different addresses, but embedding a company's employees is probably a really bad idea. ] --- ### Multiple resource references .breadcrumbs[
REST In Depth
>
Linked resources
] You can also use identifiers or hypermedia for links to multiple resources: ```json { "name": "John Doe", "addresses": [412, 633] } ``` ```json { "title": "Die Hard With a Vengeance", "_links": { "http://example.com/links/actors": [ { "href": "https://example.com/api/people/0a9duvx" }, { "href": "https://example.com/api/people/acsl9w2" }, { "href": "https://example.com/api/people/7dis92k" } ] } } ``` --- ## Pagination .breadcrumbs[
REST In Depth
] What's the problem with this HTTP request? ```http GET /phone-numbers?city=Tokyo HTTP/1.1 Host: world-phonebook.com ``` --- ### Huge collections .breadcrumbs[
REST In Depth
>
Pagination
] Some collections can grow to a point where it is not possible to get all of its elements in a single HTTP request: - It would be **too slow** - In some cases, the server might not even have enough CPU or memory **capacity** to retrieve the data You want to be able to **successively retrieve pages** of the collection:
--- ### What you need for pagination .breadcrumbs[
REST In Depth
>
Pagination
] There are two main requirements to be able to implement pagination: - The client must be able to tell the server **which elements** of the collection it wants - The server must give the client enough information to be able to **access the other elements** The most common way for clients to ask for a page is to add **URL query parameters** which the server can use to select only part of the collection: ```http GET /api/movies?`page=2`&`pageSize=50` ``` --- ### A JSON envelope .breadcrumbs[
REST In Depth
>
Pagination
] The server must also tell the client how to get the other pages. One simple way is to wrap the page in a **JSON object** (an _envelope_) that contains additional information, instead of sending a plain JSON array: ```http GET /api/movies?`page=2`&`pageSize=50` HTTP/1.1 ``` ```http HTTP/1.1 200 OK Content-Type: application/json; charset=utf-8 { `"page": 2`, `"pageSize": 50`, `"total": 231`, `"data": [` ... `]` } ``` --- #### Why an envelope? .breadcrumbs[
REST In Depth
>
Pagination
>
A JSON envelope
] - The client finds everything in **one place**, the body, the same way for every list of your API - **Browser-based clients** (web apps, hybrid mobile apps, Swagger UI) can only read a few response headers, unless the server explicitly [exposes more][http-expose-headers]. They can always read the body - It is **easy to document**: one schema in your OpenAPI documentation - With the total, the client can compute the number of pages and build a **pager**:
> Use the same envelope for every list in your API, even short ones. --- ### Other conventions you will meet .breadcrumbs[
REST In Depth
>
Pagination
] The `Link` header (see _Hyperlinks in HTTP_) gives the URLs of other pages. This is the one you saw in GitHub's API during the REST exercise: ```http Link:
; rel="next" ``` Some APIs use **custom headers** such as `X-Total-Count: 231`. Both are headers, so browser-based clients cannot read them by default. > Notice the `after` parameter in GitHub's link: GitHub uses [**cursor > pagination**][cursor-pagination]. Instead of a page number, the client sends > back a value identifying the last item it received, so that pages don't shift > when items are added. Page numbers are enough for your project. --- ### Hypermedia pagination .breadcrumbs[
REST In Depth
>
Pagination
] A hypermedia API can include the **URLs** of other pages in the envelope with standard link relations, so that the client does not have to build them: ```json { "page": 2, "pageSize": 50, "total": 231, "data": [ ... ], * "links": { * "first": "https://example.com/api/movies?page=1&pageSize=50", * "prev": "https://example.com/api/movies?page=1&pageSize=50", * "next": "https://example.com/api/movies?page=3&pageSize=50", * "last": "https://example.com/api/movies?page=5&pageSize=50" * } } ``` Hypermedia formats such as [HAL][hal] and [JSON:API][json-api] standardize the structure of these links. --- ## Resources .breadcrumbs[
REST In Depth
] - [HTTP methods][http-methods], [Web Linking][web-linking] - APIs: [Flickr][flickr-api] (L1), [AWS S3][s3-api] (L2), [Stripe][stripe-api] (L2), [GitHub][github-api] (L3), [Foxy][foxy-api] (L3) - [JSON:API][json-api], [Collection+JSON][collection-json], [HAL][hal], [JSON Hyper-Schema][json-hyper-schema], [JSON-LD][json-ld], [Siren][SIREN] - [Standards.REST: don't reinvent the wheel, use fantastic wheels][rest-standards] **Further reading** - [Architectural Styles and the Design of Network-based Software Architectures (Roy Fielding)][roy-fielding-thesis] - [The Maturity Heuristic (Leonard Richardson)][rest-maturity] - [Richardson Maturity Model (Martin Fowler)][rest-maturity-fowler] - [4 Maturity Levels Of REST API Design (Guy Levin)][rest-maturity-levin] - [How to GET a Cup of Coffee][get-coffee] - [Best Practices for Designing a Pragmatic RESTful API][pragmatic-rest] - [REST with Hypermedia - Hot or Not?][rest-with-hypermedia] --- ## Alternatives to REST .breadcrumbs[
REST In Depth
] [REST][rest] is a well-defined architectural style and industry standard that confers useful properties to your API if followed: performance, scalability, a uniform interface, portability, evolvability, etc. However, it also come with constraints. REST might not always be the best fit for all use cases. > "If all you have is a hammer, everything starts to look like a nail." **Alternatives** - Plain old RPC: [gRPC][grpc] - Query language: [GraphQL][graphql] (see the [GitHub GraphQL API][github-graphql]) - Microservices messaging for complex distributed applications: [Apache Kafka][kafka], [RabbitMQ][rabbitmq], [WAMP][wamp] **Further reading** - [Is REST still a relevant API style?][rest-still-relevant] [collection-json]: https://github.com/collection-json/spec [cursor-pagination]: https://slack.engineering/evolving-api-pagination-at-slack/ [crud]: https://en.wikipedia.org/wiki/Create,_read,_update_and_delete [flickr-api]: https://www.flickr.com/services/api/ [foxy-api]: https://api.foxycart.com/docs [get-coffee]: https://www.infoq.com/articles/webber-rest-workflow/ [github-api]: https://docs.github.com/en/rest [github-graphql]: https://docs.github.com/en/graphql [graphql]: https://graphql.org [grpc]: https://grpc.io [hal]: https://en.wikipedia.org/wiki/Hypertext_Application_Language [hal-browser]: https://github.com/mikekelly/hal-browser [hal-rfc]: https://datatracker.ietf.org/doc/html/draft-kelly-json-hal-08 [hateoas]: https://en.wikipedia.org/wiki/HATEOAS [html-form]: https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/form [http-allow]: https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Allow [http-cache]: https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/Caching [http-conditional]: https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/Conditional_requests [http-expose-headers]: https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Access-Control-Expose-Headers [http-get]: https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Methods/GET [http-idempotence]: https://developer.mozilla.org/en-US/docs/Glossary/Idempotent [http-methods]: https://www.rfc-editor.org/rfc/rfc9110.html#name-methods [http-options]: https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Methods/OPTIONS [http-partial]: https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/Range_requests [http-safe]: https://developer.mozilla.org/en-US/docs/Glossary/Safe [hypermedia]: https://en.wikipedia.org/wiki/Hypermedia [hyperlink]: https://en.wikipedia.org/wiki/Hyperlink [iana-link-relations]: https://www.iana.org/assignments/link-relations [iana-media-types]: https://www.iana.org/assignments/media-types [json-api]: https://jsonapi.org [json-hyper-schema]: https://datatracker.ietf.org/doc/draft-handrews-json-schema-hyperschema/ [json-ld]: https://json-ld.org [json-ld-spec]: https://www.w3.org/2018/jsonld-cg-reports/json-ld/ [kafka]: https://kafka.apache.org [media-type]: https://en.wikipedia.org/wiki/Media_type [npm-http-link-header]: https://www.npmjs.com/package/http-link-header [npm-format-link-header]: https://www.npmjs.com/package/format-link-header [npm-parse-link-header]: https://www.npmjs.com/package/parse-link-header [npm-search-link-header]: https://www.npmjs.com/search?q=link+header [pragmatic-rest]: https://www.vinaysahni.com/best-practices-for-a-pragmatic-restful-api [rabbitmq]: https://www.rabbitmq.com [rest]: https://en.wikipedia.org/wiki/Representational_state_transfer [rest-maturity]: https://www.crummy.com/writing/speaking/2008-QCon/act3.html [rest-maturity-fowler]: https://martinfowler.com/articles/richardsonMaturityModel.html [rest-maturity-levin]: https://blog.restcase.com/4-maturity-levels-of-rest-api-design/ [rest-standards]: https://standards.rest [rest-still-relevant]: https://nordicapis.com/is-rest-still-a-relevant-api-style/ [rest-with-hypermedia]: https://reflectoring.io/rest-hypermedia/ [roy-fielding-thesis]: https://ics.uci.edu/~fielding/pubs/dissertation/top.htm [s3-api]: https://docs.aws.amazon.com/AmazonS3/latest/API/Welcome.html [stripe-api]: https://docs.stripe.com/api [siren]: https://github.com/kevinswiber/siren [vnd-media-type]: https://en.wikipedia.org/wiki/Media_type#Vendor_tree [wamp]: https://wamp-proto.org [web-linking]: https://www.rfc-editor.org/rfc/rfc8288.html