Yo Dawg, I heard you like Discovery |
In this article I will cover several different Google technologies and how they work together, including the Google APIs Discovery Service, Google Cloud Endpoints and Dart.
I'm going to assume that you have some basic understanding of how App Engine works with Python. A little knowledge of Dart won't hurt either, but isn't really necessary if you know your way around other programming languages.
Some definitions to get us started about various technologies/projects we will be using.
The Google APIs Discovery Service is a standardized way to describe APIs in a machine-readable Discovery document, that can be used to generate client-libraries. If you have worked with any of the newer Google APIs like the Google+ REST API, the Drive API or the new YouTube v3 API you have already worked with Discovery-based APIs and will know that Google offers client libraries in a lot of different languages to access those APIs.
Google Cloud Endpoints is a new service included in Google App Engine that allows you to create your own Discovery-based APIs. The beauty in this is that you will automatically be able to use all client libraries already offered by Google (or third parties) to access your own API.
When I started to get interested in Dart and wanted to play with the Google APIs in Dart there were no functional client libaries available. So one of my first projects in Dart turned out to be a client library generator that would produce Dart client libaries based on the Discovery documents. This Discovery API Client Generator which I'm working on together with Adam Singer is already very much evolved and we'll be using it in this article to generate custom Dart client libraries for our Google Cloud Endpoints.
The Google API Discovery Check is a tool I developed a while ago that keeps track of changes in the Discovery documents of the various Google APIs. For this article I added an API to this tool that will return some meta-data about the tracked APIs.
I started off with my existing App Engine project and extended it with an API.
To create an API, you'll need to create a subclass of protorpc.remote.Service that is wrapped with the @endpoints.api decorator called with project metadata.
discovery_check.py
| from google.appengine.ext import endpoints |
| from protorpc import remote |
| @endpoints.api(name='check', version='v1', |
| description='Discovery Check API') |
| class DiscoveryCheck(remote.Service): |
| # API methods will be defined here |
To define what kind of information will be send to and from the API we define subsclasses of protorpc.messages.Message. For this example we want to request a list of APIs.
discovery_check.py
| from google.appengine.ext import endpoints |
| from protorpc import remote |
| from protorpc import messages |
| # Request message, no required parameters |
| class APIListRequest(messages.Message): |
| maxResults = messages.IntegerField(1, default=100) |
| # One API description |
| class API(messages.Message): |
| name = messages.StringField(1) |
| version = messages.StringField(2) |
| lastChange = messages.StringField(3) |
| lastCheck = messages.StringField(4) |
| firstDiscovery = messages.StringField(5) |
| # List/Array of API descriptions |
| class APIList(messages.Message): |
| items = messages.MessageField(API, 1, repeated=True) |
| @endpoints.api(name='check', version='v1', |
| description='Discovery Check API') |
| class DiscoveryCheck(remote.Service): |
| # API methods will be defined here |
Each API method in your API class will need to be wrapped with the @endpoints.method decorator, which defines what request and response messages to expect and additional parameters like the name and path of the method. The method will have to handle the request and return a response message. In our example we only define one method that fetches a list of APIs from the datastore and returns them as APIList-Message.
discovery_check.py
| from google.appengine.ext import endpoints |
| from protorpc import remote |
| from protorpc import messages |
| from db_model import ApiDoc # DB Model to access the Datastore |
| class APIListRequest(messages.Message): (...) |
| class API(messages.Message): (...) |
| class APIList(messages.Message): (...) |
| @endpoints.api(name='check', version='v1', |
| description='Discovery Check API') |
| class DiscoveryCheck(remote.Service): |
| @endpoints.method(APIListRequest, APIList, |
| path='apis', http_method='GET', |
| name='apis.list') |
| def APIList(self, request): |
| api_docs = ApiDoc.gql("ORDER BY last_check DESC") |
| items = [] |
| for api_doc in api_docs: |
| # generate API-Message from each API description |
| items.append(API(name=api_doc.key().parent().parent().name(), |
| version=api_doc.key().parent().name(), |
| lastChange=api_doc.last_change.strftime("%Y-%m-%d %H:%M"), |
| lastCheck=api_doc.last_check.strftime("%Y-%m-%d %H:%M"), |
| firstDiscovery=api_doc.first_discovery.strftime("%Y-%m-%d %H:%M"))) |
| # generate APIList-Message from array of API-Messages |
| return APIList(items=items) |
Next we'll need to define an API server that uses the API.
services.py
| from google.appengine.ext import endpoints |
| # import our API description |
| import discovery_check |
| app = endpoints.api_server([discovery_check.DiscoveryCheck], restricted=False) |
And finally we need to redirect API requests to the API Server by adding a special handler to the app.yaml.
app.yaml - new parts high-lighted, all the rest stays as it was before
| application: discovery-check |
| version: 2 |
| runtime: python27 |
| api_version: 1 |
| threadsafe: true |
| handlers: |
| - url: /images |
| static_dir: images |
| (...) |
| - url: /_ah/spi/.* |
| script: services.app |
| - url: .* |
| script: main.app |
Now we come to the fun part! After deploying everything to App Engine what you will get is your own personal Discovery API which you can access at this URL:
https://<yourapp>.appengine.com/_ah/api/discovery/v1/apis
This will list all APIs hosted by your app, including the Discovery API itself.
{ "kind": "discovery#directoryList", "discoveryVersion": "v1", "items": [ { "kind": "discovery#directoryItem", "id": "discovery:v1", "name": "discovery", "version": "v1", "title": "APIs Discovery Service", "description": "Lets you discover information about other Google APIs, such as what APIs are available, the resource and method details for each API", "discoveryRestUrl": "https://discovery-check.appspot.com/_ah/api/discovery/v1/apis/discovery/v1/rest", "discoveryLink": "./apis/discovery/v1/rest", "icons": { "x16": "http://www.google.com/images/icons/feature/filing_cabinet_search-g16.png", "x32": "http://www.google.com/images/icons/feature/filing_cabinet_search-g32.png" }, "documentationLink": "https://developers.google.com/discovery/", "preferred": true }, { "kind": "discovery#directoryItem", "id": "check:v1", "name": "check", "version": "v1", "description": "Discovery Check API", "discoveryRestUrl": "https://discovery-check.appspot.com/_ah/api/discovery/v1/apis/check/v1/rest", "discoveryLink": "./apis/check/v1/rest", "icons": { "x16": "http://www.google.com/images/icons/product/search-16.gif", "x32": "http://www.google.com/images/icons/product/search-32.gif" }, "preferred": false } ] }
We are interested in the discoveryRestUrl of our API, which includes the API description. I'm going to highlight and explain some of the more interesting features of the Discovery document.
{ "kind": "discovery#restDescription", "etag": "\"CxICGIQ83_qTKGZ2v3uQKFQ_ar8/Rw0jgK0tbZvl4LkhTfCjU4vIIXA\"", "discoveryVersion": "v1", "id": "check:v1", "name": "check", "version": "v1", "description": "Discovery Check API", "icons": { "x16": "http://www.google.com/images/icons/product/search-16.gif", "x32": "http://www.google.com/images/icons/product/search-32.gif" }, "protocol": "rest", // All method calls are relative to this baseURL "baseUrl": "https://discovery-check.appspot.com/_ah/api/check/v1/", "basePath": "/_ah/api/check/v1/", "rootUrl": "https://discovery-check.appspot.com/_ah/api/", "servicePath": "check/v1/", "batchPath": "batch", "parameters": { // general query parameters, not really interesting for our project // ... }, // Schemas correlate to the Message classes we defined "schemas": { // Compare to API class "DiscoveryCheckAPI": { "id": "DiscoveryCheckAPI", "type": "object", "properties": { "firstDiscovery": {"type": "string"}, "lastChange": {"type": "string"}, "lastCheck": {"type": "string"}, "name": {"type": "string"}, "version": {"type": "string"} } }, // Compare to APIList class "DiscoveryCheckAPIList": { "id": "DiscoveryCheckAPIList", "type": "object", "properties": { "items": { "type": "array", "items": { "$ref": "DiscoveryCheckAPI" } } } } }, // Resources include all the methods the API supports. // Since we named our method apis.list we have an "apis" resource with a "list" method "resources": { "apis": { "methods": { // Compare to APIList method "list": { "id": "check.apis.list", // The full path to request the list will be baseUrl+path "path": "apis", "httpMethod": "GET", // Parameters we defined in the APIListRequest class "parameters": { "maxResults": { "type": "string", "default": "100", "format": "int64", "location": "query" } }, // This tells us that we can expect an APIList as result of the request "response": {"$ref": "DiscoveryCheckAPIList"} } } } } }
Stitching together the baseUrl and the method path we get this URL to request the API list:
https://discovery-check.appspot.com/_ah/api/check/v1/apis
Opening this URL in a browser gives us the expected result:
{ "items": [ { "lastCheck": "2013-01-31 16:50", "lastChange": "2012-11-16 04:51", "version": "v1", "name": "drive", "firstDiscovery": "2012-04-24 15:12", "kind": "check#apisItem" }, { "lastCheck": "2013-01-31 16:00", "lastChange": "2012-10-10 18:11", "version": "v3", "name": "youtube", "firstDiscovery": "2012-06-27 01:33", "kind": "check#apisItem" }, ... }
Now that we have the API up and running we of course want to use it in an application, and in this case we're going to create a Dart web application.
To do this we will first create a Dart client library using the Discovery API Dart Client Generator. This generator allows us to supply the URL to a discovery document with a --url parameter so we're going to generate our client-library like this:
dart bin/generate.dart --url=https://discovery-check.appspot.com/_ah/api/discovery/v1/apis/check/v1/rest
This will create a fully functional client library (for web and console applications) in the output/dart_check_v1_api_client folder. Normally you would now publish this library (f.e. on github) and include it in your web application as dependency. Since we only want to test the functionality we're going to open the library in the DartEditor and add an example folder with a simple demo application.
example/demo.html
<!DOCTYPE html> <html> <head> <meta charset="utf-8"> <title>Discovery Check API Demo</title> <link rel="stylesheet" href="demo.css"> </head> <body> <h1>Discovery Check API Demo</h1> <-- Just a div to display the API response --> <div id="text">Loading, please wait...</div> <-- Our dart application --> <script type="application/dart" src="demo.dart"></script> <-- Some magic to make Dart work --> <script src="packages/browser/dart.js"></script> </body> </html>
example/demo.dart
import "dart:html"; // this is our generated library import "package:check_v1_api/check_v1_api_browser.dart" as checklib; void main() { var container = query("#text"); // initialize the client library var check = new checklib.Check(); // call the apis.list method expecting an APIList in response check.apis.list().then((checklib.DiscoveryCheckAPIList data) { container.text = ""; // display the response data data.items.forEach((item) { container.appendHtml("${item.name} ${item.version} - ${item.lastChange}<br>"); }); }) .catchError((e) { container.appendHtml("$e<br>"); return true; }); }
This will create a page listing all the APIs currently being tracked by Discovery-Check with their latest change date.
Discoveries are fun and now you should be able to discover discoveries while discovering discoveries ;)
If you have any further questions feel free to contact me on Google+
You can find the generated library including the example on GitHub
This work is licensed under a Creative Commons Attribution 3.0 Unported License.