Designing-a-Beautiful-REST+JSON-API

1. Beautiful REST+JSON APIs Les Hazlewood @lhazlewood Apache Shiro Project Chair Expert Group Member, JEE Application Security CTO, Stormpath, stormpath.com

2. .com •  User Management API for Developers •  Password security •  Authentication and Authorization •  LDAP & Active Directory Cloud Sync •  Instant-on, scalable, and highly available •  Free for developers @lhazlewood

3. Outline •  APIs, REST & JSON •  REST Fundamentals •  Design Base URL Versioning Resource Format Return Values Content NegoEaEon References (Linking) PaginaEon Query Parameters AssociaEons @lhazlewood Errors IDs Method Overloading Resource Expansion ParEal Responses Caching & Etags Security MulE Tenancy Maintenance Batch OperaEons

4. APIs •  •  •  •  •  Applications Developers Pragmatism over Ideology Adoption Scale @lhazlewood

5. Why REST? •  •  •  •  •  •  Scalability Generality Independence Latency (Caching) Security Encapsulation @lhazlewood

6. Why JSON? •  •  •  •  •  Ubiquity Simplicity Readability Scalability Flexibility @lhazlewood

7. HATEOAS •  •  •  •  •  •  •  Hypermedia As The Engine Of Application State @lhazlewood

8. REST Is Easy @lhazlewood

9. REST Is *&@#$! Hard (for providers) @lhazlewood

10. REST can be easy (if you follow some guidelines) @lhazlewood

11. Example Domain: Stormpath •  •  •  •  •  •  Applications Directories Accounts Groups Associations Workflows

12. Fundamentals @lhazlewood

13. Resources Nouns, not Verbs Coarse Grained, not Fine Grained Architectural style for use-case scalability @lhazlewood

14. What If? /getAccount /createDirectory /updateGroup /verifyAccountEmailAddress @lhazlewood

15. What If? /getAccount /getAllAccounts /searchAccounts /createDirectory /createLdapDirectory /updateGroup /updateGroupName /findGroupsByDirectory /searchGroupsByName /verifyAccountEmailAddress /verifyAccountEmailAddressByToken … Smells like bad RPC. DON’T DO THIS. @lhazlewood

16. Keep It Simple @lhazlewood

17. The Answer Fundamentally two types of resources: Collection Resource Instance Resource @lhazlewood

18. Collection Resource /applications @lhazlewood

19. Instance Resource /applications/a1b2c3 @lhazlewood

20. Behavior •  GET •  PUT •  POST •  DELETE •  HEAD @lhazlewood

21. Behavior POST, GET, PUT, DELETE ≠ 1:1 Create, Read, Update, Delete @lhazlewood

22. Behavior As you would expect: GET = Read DELETE = Delete HEAD = Headers, no Body @lhazlewood

23. Behavior Not so obvious: PUT and POST can both be used for Create and Update @lhazlewood

24. PUT for Create IdenEﬁer is known by the client: PUT /applications/clientSpecifiedId { … } @lhazlewood

25. PUT for Update Full Replacement PUT /applications/existingId { “name”: “Best App Ever”, “description”: “Awesomeness” } @lhazlewood

26. PUT Idempotent @lhazlewood

27. POST as Create On a parent resource POST /applications { “name”: “Best App Ever” } Response: 201 Created Location: https://api.stormpath.com/applications/a1b2c3 @lhazlewood

28. POST as Update On instance resource POST /applications/a1b2c3 { “name”: “Best App Ever. Srsly.” } Response: 200 OK @lhazlewood

29. POST NOT Idempotent @lhazlewood

30. Media Types •  Format SpeciﬁcaEon + Parsing Rules •  Request: Accept header •  Response: Content-Type header •  •  •  •  application/json application/foo+json application/foo+json;application … @lhazlewood

31. Design Time! @lhazlewood

32. Base URL @lhazlewood

33. http(s)://foo.io vs http://www.foo.com/dev/service/api/rest @lhazlewood

34. http(s)://foo.io Rest Client vs Browser @lhazlewood

35. Versioning @lhazlewood

36. URL https://api.stormpath.com/v1 vs. Media-Type application/foo+json;application&v=2 application/foo2+json;application @lhazlewood

37. Resource Format @lhazlewood

38. Media Type Content-Type: application/json When time allows: application/foo+json application/foo2+json;bar=baz … @lhazlewood

39. Media Type Don’t go overboard! Media Type != Schema! Most only need 2 or 3 custom media types: •  instance resource •  collection resource application/foo+json application/foo2+json;bar=baz … @lhazlewood

40. camelCase ‘JS’ in ‘JSON’ = JavaScript myArray.forEach Not myArray.for_each account.givenName Not account.given_name Underscores for property/function names are unconventional for JS. Stay consistent. @lhazlewood

41. Date/Time/Timestamp There’s already a standard. Use it: ISO 8601 Example: { …, “createdAt”: “2013-07-10T18:02:24.343Z”, ... } Use UTC! @lhazlewood

42. createdAt / updatedAt @lhazlewood

43. createdAt / updatedAt Most people will want this at some point { …, “createdAt”: “2013-07-10T18:02:24.343Z”, “updatedAt”: “2014-09-29T07:02:48.761Z” } Use UTC! @lhazlewood

44. Response Body @lhazlewood

45. GET obvious What about POST? Return the representation in the response when feasible. Add override (?_body=false) for control @lhazlewood

46. Content Negotiation @lhazlewood

47. Header •  Accept header •  Header values comma delimited •  q param determines precedence, defaults to 1, then conventionally by list order GET /applications/a1b2c3 Accept: application/json, text/ plain;q=0.8 @lhazlewood

48. Resource Extension /applications/a1b2c3.json /applications/a1b2c3.csv … ConvenEonally overrides Accept header @lhazlewood

49. HREF •  Distributed Hypermedia is paramount! •  Every accessible Resource has a canonical unique URL •  Replaces IDs (IDs exist, but are opaque). •  Critical for linking, as we’ll soon see @lhazlewood

50. Instance w/ HREF (v1) GET /accounts/x7y8z9 200 OK { “href”: “https://api.stormpath.com/v1/accounts/x7y8z9”, “givenName”: “Tony”, “surname”: “Stark”, ... } @lhazlewood

51. Resource References aka ‘Linking’ (v1) @lhazlewood

52. •  Hypermedia is paramount. •  Linking is fundamental to scalability. •  Tricky in JSON •  XML has it (XLink), JSON doesn’t •  How do we do it? @lhazlewood

53. Instance Reference (v1) GET /accounts/x7y8z9 200 OK { “href”: “https://api.stormpath.com/v1/accounts/x7y8z9”, “givenName”: “Tony”, “surname”: “Stark”, …, “directory”: ???? } @lhazlewood

54. Instance Reference (v1) GET /accounts/x7y8z9 200 OK { “href”: “https://api.stormpath.com/v1/accounts/x7y8z9”, “givenName”: “Tony”, “surname”: “Stark”, …, “directory”: { “href”: “https://api.stormpath.com/v1/directories/g4h5i6” } } @lhazlewood

55. Collection Reference (v1) GET /accounts/x7y8z9 200 OK { “href”: “https://api.stormpath.com/v1/accounts/x7y8z9”, “givenName”: “Tony”, “surname”: “Stark”, …, “groups”: { “href”: “https://api.stormpath.com/v1/accounts/x7y8z9/groups” } } @lhazlewood

56. Linking v2 (recommended) @lhazlewood

57. Instance HREF (v2) GET /accounts/x7y8z9 200 OK { “meta”: { “href”: “https://api.stormpath.com/v1/accounts/x7y8z9”, “mediaType”: “application/ion+json”, ... }, “givenName”: “Tony”, “surname”: “Stark”, … } @lhazlewood

58. Instance Reference (v2) GET /accounts/x7y8z9 200 OK { “meta”: { ... }, “givenName”: “Tony”, “surname”: “Stark”, …, “directory”: { “meta”: { “href”: “https://api.stormpath.com/v1/directories/g4h5i6” “mediaType”: “application/ion+json” } } } @lhazlewood

59. Collection Reference (v2) GET /accounts/x7y8z9 200 OK { “meta”: { ... }, “givenName”: “Tony”, “surname”: “Stark”, …, “groups”: { “meta”: { “href”: “https://api.stormpath.com/v1/accounts/x7y8z9/groups”, “mediaType”: “application/ion+json”, “rel”: [“collection”] } } } @lhazlewood

60. Reference Expansion (aka Entity Expansion, Link Expansion) @lhazlewood

61. Account and its Directory? @lhazlewood

62. GET /accounts/x7y8z9?expand=directory 200 OK { “meta”: {...}, “givenName”: “Tony”, “surname”: “Stark”, …, “directory”: { “meta”: { ... }, “name”: “Avengers”, “description”: “Hollywood’s hope for more $”, “createdAt”: “2012-07-01T14:22:18.029Z”, … } } @lhazlewood

63. Partial Representations @lhazlewood

64. GET /accounts/x7y8z9? fields=givenName,surname,directory(name) @lhazlewood

65. Collections! @lhazlewood

66. Collections •  •  •  •  A first class resource ‘citizen’ Own href / metadata Own properties Different from all other collections @lhazlewood

67. GET /accounts/x7y8z9/groups 200 OK { “meta”: { ... }, “offset”: 0, “limit”: 25, “size”: 289, “first”: { “meta”:{“href”: “…/accounts/x7y8z9/groups?offset=0”}}, “previous”: null, “next”: { “meta”:{“href”: “…/accounts/x7y8z9/groups?offset=25”}}, “last”: { “meta”:{“href”: “…”}}, “items”: [ { “meta”: { “href”: “…”, ...} }, … ] } @lhazlewood

68. Pagination @lhazlewood

69. Collection Resource supports query params: •  Offset •  Limit …/applications?offset=50&limit=25 @lhazlewood

70. GET /accounts/x7y8z9/groups 200 OK { “meta”: { ... }, “offset”: 0, “limit”: 25, “first”: { “meta”:{“href”: “…/accounts/x7y8z9/groups?offset=0”}}, “previous”: null, “next”: { “meta”:{“href”: “…/accounts/x7y8z9/groups?offset=25”}}, “last”: { “meta”:{“href”: “…”}}, “items”: [ { “meta”: { “href”: “…”, ...} }, { “meta”: { “href”: “…”, ...} }, … ] } @lhazlewood

71. Sorting @lhazlewood

72. GET .../accounts? orderBy=surname,givenName%20desc @lhazlewood

73. Search @lhazlewood

74. “Find all accounts with a ‘company.com’ email address that can login to a particular application” @lhazlewood

75. GET /applications/x7y8z9/accounts? email=*company.com 200 OK { “meta”: { ... }, “offset”: 0, “limit”: 25, “first”: { “meta”:{“href”: “/applications/x7y8z9/accounts? email=*company.com&offset=0”}}, “previous”: null, “next”: { “meta”:{“href”: “/applications/x7y8z9/accounts? email=*company.com&offset=25”}}, “last”: { “meta”:{“href”: “…”}}, “items”: [ { “meta”: { “href”: “…”, ...} }, { “meta”: { “href”: “…”, ...} }, … ] } @lhazlewood

76. Search cont’d •  Filter search .../accounts?q=some+value •  Attribute Search .../accounts? surname=Joe&email=*company.com @lhazlewood

77. Search cont’d •  Starts with ?email=joe* •  Ends with ?email=*company.com •  Contains ?email=*foo* @lhazlewood

78. Search cont’d •  Range queries “all accounts created between September 1st and the 15th” .../accounts? createdAt=[2014-09-01,2014-09-15] @lhazlewood

79. Many To Many @lhazlewood

80. Group to Account •  A group can have many accounts •  An account can be in many groups •  Each mapping is a resource: GroupMembership @lhazlewood

81. GET /groupMemberships/23lk3j2j3 200 OK { “meta”:{“href”: “…/groupMemberships/23lk3j2j3”}, “account”: { “meta”:{“href”: “…”} }, “group”: { “meta”{“href”: “…”} }, … } @lhazlewood

82. GET /accounts/x7y8z9 200 OK { “meta”:{“href”: “…/accounts/x7y8z9”}, “givenName”: “Tony”, “surname”: “Stark”, …, “groups”: { “meta”:{“href”: “…/accounts/x7y8z9/groups”} }, “groupMemberships”: { “meta”:{“href”: “…/groupMemberships?accountId=x7y8z9”} } } @lhazlewood

83. Async or Long-Lived Operations @lhazlewood

84. POST /emails { “from”: me@somewhere.com, “subject”: “Hi!” “body”: “...” } @lhazlewood

85. 204 Accepted Location: /emails/23Sd932sSl { “status”: “queued”, ... } @lhazlewood

86. GET /emails/23Sd932sSl Expires: 2014-09-29T18:00:00.000Z { “status”: “sent”, ... } @lhazlewood

87. Batch Operations @lhazlewood

88. •  Each batch reflects a resource •  Batches are likely to be a collection •  Batches are likely to have a status •  Batch deletes easier than create/update @lhazlewood

89. Batch Delete “Delete all company.com accounts” DELETE /accounts? email=*@company.com @lhazlewood

90. Batch Create / Update Already have a CollecEon concept. Use it. @lhazlewood

91. Batch Create or Update POST /accounts { “items”: [ { ... account 1 ... }, { ... account 2 ... }, ... ] } @lhazlewood

92. Batch Operations: The ‘Catch’ Caching is bypassed enErely L @lhazlewood

93. 204 Accepted Location: /batches/a1b2c3 { “status”: “processing”, //overall status “size”: “n”, “limit”: 25, ..., “items”: { { response 1 (w/ individual status) ...}, { response 2 (w/ individual status) ...}, ... } } @lhazlewood

94. Errors @lhazlewood

95. •  As descriptive as possible •  As much information as possible •  Developers are your customers @lhazlewood

96. POST /directories 409 Conflict { “status”: 409, “code”: 40924, “property”: “name”, “message”: “A Directory named ‘Avengers’ already exists.”, “developerMessage”: “A directory named ‘Avengers’ already exists. If you have a stale local cache, please expire it now.”, “moreInfo”: “https://www.stormpath.com/docs/ api/errors/40924” } @lhazlewood

97. Security @lhazlewood

98. Avoid sessions when possible Authenticate every request if necessary Stateless Authorize based on resource content, NOT URL! Use Existing Protocol: Oauth 1.0a, Oauth2, Basic over SSL only Custom Authentication Scheme: Only if you provide client code / SDK Only if you really, really know what you’re doing Use API Keys instead of Username/Passwords @lhazlewood

99. 401 vs 403 •  401 “Unauthorized” really means Unauthenticated “You need valid credentials for me to respond to this request” •  403 “Forbidden” really means Unauthorized “Sorry, you’re not allowed!” @lhazlewood

100. HTTP Authentication Schemes •  Server response to issue challenge: WWW-Authenticate: <scheme name> realm=“Application Name” •  Client request to submit credenEals: Authorization: <scheme name> <data> @lhazlewood

101. API Keys •  •  •  •  •  •  •  Entropy Password Reset Independence Scope Speed Limited Exposure Traceability @lhazlewood

102. IDs @lhazlewood

103. •  IDs should be opaque •  Should be globally unique •  Avoid sequential numbers (contention, fusking) •  Good candidates: UUIDs, ‘Url64’ @lhazlewood

104. HTTP Method Overrides @lhazlewood

105. POST /accounts/x7y8z9?_method=DELETE @lhazlewood

106. Caching & Concurrency Control @lhazlewood

107. Server (iniEal response): ETag: "686897696a7c876b7e” Client (later request): If-None-Match: "686897696a7c876b7e” Server (later response): 304 Not Modified @lhazlewood

108. Maintenance @lhazlewood

109. Use HTTP Redirects Create abstraction layer / endpoints when migrating Use well defined custom Media Types @lhazlewood

110. .com •  •  •  •  •  •  •  Free for developers Eliminate months of development Automatic security best practices Single Sign On for your apps API Authentication & Key Management Token Authentication for SPAs / Mobile Authorization Libraries and integraEons: h`ps://docs.stormpath.com @lhazlewood