From 406fee5458eca3963100818f6c4d6af11ac561c6 Mon Sep 17 00:00:00 2001 From: "Elf M. Sternberg" Date: Fri, 30 Mar 2018 20:00:15 -0700 Subject: [PATCH 1/3] Finishing up the documentation. Next step: Blogging. --- docs/Part_02.md | 61 +++++++++++++++ docs/Part_03.md | 202 ++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 263 insertions(+) create mode 100644 docs/Part_03.md diff --git a/docs/Part_02.md b/docs/Part_02.md index 4b2ee5f..27922bb 100644 --- a/docs/Part_02.md +++ b/docs/Part_02.md @@ -32,6 +32,7 @@ them into a new file inside the `timeofday/` folder. You will also have to create a package name and import any packages being used. Now your file, which I've called `timeofday/handlers.go`, looks like this: +``` <>= package timeofday @@ -48,15 +49,18 @@ func PostTime(params operations.TimePostParams) middleware.Responder { return middleware.NotImplemented("operation .TimePost has not yet been implemented") } @ +``` And now go back to `restapi/configure_timeofday.go`, add `github.com/elfsternberg/timeofday/clock` to the imports, and change the handler lines to look like this: +``` <>= api.TimeGetHandler = operations.TimeGetHandlerFunc(timeofday.GetTime) api.TimePostHandler = operations.TimePostHandlerFunc(timeofday.PostTime) @ +``` ## Implementation @@ -90,6 +94,7 @@ and then return either a success message or an error message. Looking in the operations files, there are a methods for good and bad returns, as we described in the swagger file. +``` <>= func GetTime(params operations.TimeGetParams) middleware.Responder { var tz *string = nil @@ -101,6 +106,7 @@ func GetTime(params operations.TimeGetParams) middleware.Responder { thetime, err := getTimeOfDay(params.Timezone) @ +``` The first thing to notice here is the `params` field: we're getting a customized, tightly bound object from the server. There's no hope of @@ -113,6 +119,7 @@ We then call a (thus far undefined) function called `getTimeOfDay`. Let's deal with the error case: +``` <>= if err != nil { return operations.NewTimeGetNotFound().WithPayload( @@ -122,6 +129,7 @@ Let's deal with the error case: }) } @ +``` That's a lot of references. We have a model, an operation, and what's that "swag" thing? In order to satisfy Swagger's strictness, we use @@ -134,6 +142,7 @@ the response with content. The good path is similar: +``` <>= return operations.NewClockGetOK().WithPayload( &models.Timeofday{ @@ -141,8 +150,60 @@ The good path is similar: }) } @ +``` Now might be a good time to go look in `models/` and `/restapi/options`, to see what's available to you. You'll need to do so anyway, because unless you go to the [git repository](https://github.com/elfsternberg/go-swagger-tutorial) +and cheat, I'm going to leave it up to you to implement the PostTime(). + +There's still one thing missing, though: the actual time of day. We'll +need a default, and we'll need to test to see if the default is needed. +The implementation is straightforward: + +``` +<>= +func getTimeOfDay(tz *string) (*string, error) { + defaultTZ := "UTC" + + t := time.Now() + if tz == nil { + tz = &defaultTZ + } + + utc, err := time.LoadLocation(*tz) + if err != nil { + return nil, errors.New(fmt.Sprintf("Time zone not found: %s", *tz)) + } + + thetime := t.In(utc).String() + return &thetime, nil +} +@ +``` + +Now, if you've written everything correctly, and the compiler admits +that you have (or you can cheat and download the 0.2.0-tagged version +from the the repo), you'll be able to build, compile, and run the +server, and see it working: + +``` +$ go build ./cmd/timeofday-server/ +$ ./timeofday-server --port=8080 +``` + +And then test it with curl: + +``` +$ curl 'http://localhost:8020/timeofday/v1/time' +{"timeofday":"2018-03-31 02:57:48.814683375 +0000 UTC"} +$ curl 'http://localhost:8020/timeofday/v1/time?timezone=UTC' +{"timeofday":"2018-03-31 02:57:50.443200906 +0000 UTC"} +$ curl 'http://localhost:8020/timeofday/v1/time?timezone=America/Los_Angeles' +{"timeofday":"2018-03-30 19:57:59.886650128 -0700 PDT"} +``` + +And that's the end of Part 2. If you've gotten this far, +congratulations! On to [Part 3](TK:). + diff --git a/docs/Part_03.md b/docs/Part_03.md new file mode 100644 index 0000000..31af0cd --- /dev/null +++ b/docs/Part_03.md @@ -0,0 +1,202 @@ +# Adding Command Line Arguments to the Swagger Server + +The first two parts of my swagger tutorial were dedicated to the +straightforward art of getting swagger up and running. While I hope +they're helpful, the whole point of those was to get you to the point +where you had the Timezone project, so I could teach how to extend the +Command Line Arguments of a swagger project. + +Nobody else, so far as I know, knows how to do this. One thing that I +emphasized in [Go Swagger Part 2](TK:) was that `configure_timeofday.go` +is the *only* file you should be touching, it's the interface between +the server and your business logic. Every example of adding new flags +to the command line, even the ones provided by the GoSwagger authors, +starts by modifying the file `cmd/\-server/main.go`, one of +those files clearly marked `// DO NOT EDIT`. + +We're not going to edit files marked `// DO NOT EDIT`. + +To understand the issue, though, we have to understand the tool swagger +uses for handling command line arguments, `go-flags`. + +## Go-Flags + +`go-flags` is the tool Swagger uses by default for handling command line +arguments. It's a clever tool that uses Go's +[tags and retrospection](TK:) features to encade the details of the CLI +directly into a structure that will hold the options passed in on the +command line. + +## The implementation + +We're going to add a single feature: the default timezone. In Part 2, +we hard-coded the default timezone into the handler, but what if we want +to change the default timezone more readily than recompiling the binary +every time? The Go, Docker, and Kubernetes crowd argue that that's +acceptable, but I still want more flexibility. + +To start, we're going to open a new file it the folder with our handlers +and add a new file, `timezone.go`. We're going to put a single tagged +structure with a single field to hold our timezone CLI argument, and +we're going to use `go-flags` protocol to describe our new command line +flag. + +Here's the whole file: + +``` +<>= +package timeofday +type Timezone struct { + Timezone string `long:"timezone" short:"t" description:"The default time zone" env:"GOTIME_DEFAULT_TIMEZONE" default:"UTC"` +} +@ +``` + +If you want to know what you can do with `go-flags`, open the file +`./restapi/server.go` and examine the Server struct there, and compare +its contents to what you see when you type `timeofday-server --help`. +You can learn a lot by reading even the generated source code. As +always, `// DO NOT EDIT THIS FILE`. + +Next, go into `configure_timeofday.go`, and find the function +`configureFlags`. This, unsurprisingly, is where this feature is +*supposed* to go. + +We've already imported the `timeofday` package, so we have access to +our new Timezone type. Right above `configureFlags`, let's create an +instance of this struct and populate it with defaults: + +``` +<>= +var Timezone = timeofday.Timezone{ + Timezone: "UTC", +} +@ +``` + +See the comment in configureFlags? Note that `swag` package? You'll +have to add it to the imports. It should be already present, as it came +with the rest of the swagger installation. Just add: + +``` +<>= + swag "github.com/go-openapi/swag" +@ +``` + +And now modify `configureFlags()`: + +``` +<>= +func configureFlags(api *operations.TimeofdayAPI) { + api.CommandLineOptionsGroups = []swag.CommandLineOptionsGroup{ + swag.CommandLineOptionsGroup{ + ShortDescription: "Time Of Day Service Options", + Options: &Timezone, + }, + } +} +@ +``` + +See that `ShortDescription` there? When you run the `--help` option for +the server, you'll see a section labeled "Application Options", and +another labeled "Help Options". We're adding a new section, "Service +Options", which will include our customizations. This conceptually +allows us to distinguish between routine options of a microservice, and +the *specific* options of *this* microservice. + +Always distinguish between your framework and your business logic. +(I've often seen this written as "always distinguish between execution +exceptions and business exceptions," and it's great advice is similarly +here.) + +You can now build your server (`go build ./cmd/timeofday-server/`), and +run it (`./timeofday-server --help`), and you'll see your new options. +Of course, they don't do anything, we haven't modified your business +logic! + +## The Context Problem + +This is where most people have a problem. How do the values that now +populate the Timezone struct make their way down to the handlers? There +are a number of ways to do this. The "edit `main.go`" people just make +it a global variable available to the whole server, but I'm here to tell +you doing so is sad and you should feel sad if you do it. What we have +here, in our structure that holds our CLI options, is a *context*. How +do we set the context? + +The *correct* way is to modify the handlers so they have the context +when they're called upon. The way we do that is via the oldest +object-oriented technique of all time, one that dates all the way back +to 1959 and the invention of Lisp: *closures*. + +Remember these lines in `configure_timeofday.go`, from way back? + +``` + api.TestGetHandler = operations.TestGetHandlerFunc(func(params operations.TestGetParams) middleware.Responder { + return middleware.NotImplemented("operation .TestGet has not yet been implemented") + }) +``` + +See that function that actually gets passed to TestHandlerGetFunc()? +It's anonymous. We broke it out and gave it a name and stuff and filled +it out with business logic and made it work. We're going to go back and +replace those lines, *again*, so they look like this: + +``` + api.TimeGetHandler = operations.TimeGetHandlerFunc(timeofday.GetTime(&Timezone)) + api.TimePostHandler = operations.TimePostHandlerFunc(timeofday.PostTime(&Timezone)) +``` + +Those are no longer references to functions. They're *function calls*! +What do those functions return? Well, we know TimeGetHandlerFunc() is +expecting a function, so so that function call had better return a +function. + +And indeed it does. + +``` +func GetTime(timezone *Timezone) func(operations.TimeGetParams) middleware.Responder{ + defaultTZ := timezone.Timezone + + return func(params operations.TimeGetParams) middleware.Responder { + // Everything else is the same... except we need *two* levels of + // closing } at the end! +``` + +Now, instead of returning a function defined at compile time, we +returned a function that is finalized when GetTime() is called, and it +now holds a permanent reference to our Timezone object. Do the same +thing for `PostTime`. + +There's one more thing we have to do. We've moved our default timezone +to the `configure_timeofday.go` file, so we don't need it here anymore: + +``` +func getTimeOfDay(tz *string) (*string, error) { + t := time.Now() + utc, err := time.LoadLocation(*tz) + if err != nil { + return nil, errors.New(fmt.Sprintf("Time zone not found: %s", *tz)) + } + + thetime := t.In(utc).String() + return &thetime, nil +} +``` + +And that's it. That's everything. You can add all the command line +arguments you want, and only preserve the fields that are relevant to +the particular handler you're going to invoke. + +And *that's* how you add command line arguments to Swagger servers +correctly. + + + + + + + + From 0237bc1d8ac4ec64f2bcd1da76032a4f5f250d69 Mon Sep 17 00:00:00 2001 From: "Elf M. Sternberg" Date: Fri, 30 Mar 2018 22:26:12 -0700 Subject: [PATCH 2/3] Documentation done, and Readme annotated with the appropriate links. --- docs/Part_01.md | 44 +++++++++++++++++++++---- docs/Part_02.md | 35 +++++++++++++------- docs/Part_03.md | 85 ++++++++++++++++++++++++++++++++++--------------- 3 files changed, 120 insertions(+), 44 deletions(-) diff --git a/docs/Part_01.md b/docs/Part_01.md index 3beafe8..471be1f 100644 --- a/docs/Part_01.md +++ b/docs/Part_01.md @@ -11,6 +11,23 @@ But if you're writing microservices for enterprise customers, yes, you should use Go, and yes, you should use OpenAPI and Swagger. So here's how it's done. +All of the files for this tutorial are available from the +elfsternberg/go-swagger-tutorial repo at github. There are *two* phases +to this tutorial, and the first phase is the base Go Swagger +implementation. I strongly recommend that if you're going to check out +the source code in its entirety, that you start with the +[Basic Version](https://github.com/elfsternberg/go-swagger-tutorial/tree/0.2.0), +and only check out the +[Advanced version](https://github.com/elfsternberg/go-swagger-tutorial/tree/0.4.0) +when you get to Part 3. + +Just be aware that if you see stuff that looks like `<>`, or a +single `@` alone on a line, that's just part of my code layout; do *not* +include those in your source code, they're not part of Go or Swagger. +Sorry about that. + +## Go Swagger! + [Swagger](https://swagger.io/) is a specification that describes the ndpoints for a webserver's API, usually a REST-based API. HTTP uses verbs (GET, PUT, POST, DELETE) and endpoints (/like/this) to describe @@ -36,12 +53,12 @@ functions with your business logic. There are three things that are your responsibility: 1. Write the specification that describes *exactly* what the server -accepts as requests and returns as responses. +accepts as requests and returns as responses, and generate a server from +this specification. 2. Write the business logic. -3. Glue the business logic into the server generated from the -specification. +3. Glue the business logic into the generated server. In Go-Swagger, there is *exactly one* file in the generated code that you need to change. Every other file is labeled "DO NOT EDIT." This @@ -64,6 +81,7 @@ $ go get -u github.com/golang/dep/cmd/dep $ go get -u github.com/go-swagger/go-swagger/cmd/swagger ``` + ## Initialization Now you're going to create a new project. Do it in your src directory @@ -91,15 +109,18 @@ optional timezone as a JSON argument in the body of the POST. First, let's version our API. You do that with Basepaths: +``` <>= basePath: /timeofday/v1 @ +``` Now that we have a base path that versions our API, we want to define our endpoint. The URL will ultimately be `/timeofday/v1/time`, and we want to handle both GET and POST requests, and our responses are going to be **Success: Time of day** or **Timezone Not Found**. +``` <>= paths: /time: @@ -133,11 +154,13 @@ paths: schema: $ref: "#/definitions/NotFound" @ +``` The `$ref` entries are a YAML thing for referring to something else. -The octothorpe symbol `(#)` indicates "look in the current file. So +The octothorpe symbol `(#)` indicates "look in the current file." So now we have to create those paths: +``` <>= definitions: NotFound: @@ -158,6 +181,7 @@ definitions: properties: TimeOfDay: string @ +``` This is *really verbose*, but on the other hand it is *undeniably complete*: these are the things we take in, and the things we respond @@ -165,6 +189,7 @@ with. So now your file looks like this: +``` <>= swagger: "2.0" info: @@ -186,14 +211,19 @@ schemes: <> @ +``` Now that you have that, it's time to generate the server! `$ swagger generate server -f swagger.yml` It will spill out the actions it takes as it generates your new REST -server. **Do not** follow the advice at the end of the output. -There's a better way. +server. **Do not** follow the advice at the end of the output. There's +a better way. Use `dep`, which will automagically find all your +dependencies for you, download them to a project-specific `vendor/` +folder, and _lock_ the specific commit in the record so version creep +won't break your project in the future. `dep` has become even Google's +recommended dependency control mechanism. Just run: `$ dep init` @@ -224,4 +254,4 @@ $ curl http://localhost:8082/timeofday/v1/time Congratulations! You have a working REST server that does, well, nothing. -For part two, we'll make our server actually do things. +For Part 2, we'll make our server actually do things. diff --git a/docs/Part_02.md b/docs/Part_02.md index 27922bb..569d04c 100644 --- a/docs/Part_02.md +++ b/docs/Part_02.md @@ -1,36 +1,45 @@ # Review of Part One -In [Part One of Go-Swagger](TK:), we generated a on OpenAPI 2.0 server -with REST endpoints. The server builds and responds to queries, but -every valid query ends with "This feature has not yet been -implemented." +In +[Part One of Go-Swagger](http://www.elfsternberg.com/2018/03/30/writing-microservice-swagger-part-1-specification/), +we generated a on OpenAPI 2.0 server with REST endpoints. The server +builds and responds to queries, but every valid query ends with "This +feature has not yet been implemented." It's time to implement the feature. I want to emphasize that with Go Swagger there is *only* one generated -file you need to touch. Since our project is named `timezone`, the -file will be named `restapi/configure_timezone.go`. Our first step +file you need to touch. Since our project is named `timeofday`, the +file will be named `restapi/configure_timeofday.go`. Our first step will be to break those "not implemented" functions out into their own Go package. That package will be our business logic. The configure file and the business logic package will be the *only* things we change. +A reminder: The final source code for this project is available on +Github, however Parts One & Two deal with the most common +implementation, a server with hard-coded default values. For these +chapters, please consult +[that specific version of the code](https://github.com/elfsternberg/go-swagger-tutorial/tree/0.2.0). + ## Break out the business logic Create a new folder in your project root and call it `timeofday`. Open up your editor and find the file `restapi/configure_timeofday.go`. In your `swagger.yml` file you created two endpoints and gave them each -an `operationId`: `TimekPost` and `TimeGet`. Inside +an `operationId`: `TimePost` and `TimeGet`. Inside `configure_timeofday.go`, you should find two corresponding assignments in the function `configureAPI()`: `TimeGetHandlerFunc` and -`ClockPostHandlerFunc`. Inside those function calls, you'll find +`TimePostHandlerFunc`. Inside those function calls, you'll find anonymous functions. I want you to take those anonymous functions, cut them out, and paste them into a new file inside the `timeofday/` folder. You will also have to create a package name and import any packages being used. Now your -file, which I've called `timeofday/handlers.go`, looks like this: +file, which I've called `timeofday/handlers.go`, looks like this (note +that you'll have to change your import paths as you're probably not +elfsternberg. Heck, _I'm_ probably not elfsternberg): ``` <>= @@ -155,7 +164,7 @@ The good path is similar: Now might be a good time to go look in `models/` and `/restapi/options`, to see what's available to you. You'll need to do so anyway, because unless you go to the -[git repository](https://github.com/elfsternberg/go-swagger-tutorial) +[git repository](https://github.com/elfsternberg/go-swagger-tutorial/tree/0.2.0) and cheat, I'm going to leave it up to you to implement the PostTime(). There's still one thing missing, though: the actual time of day. We'll @@ -205,5 +214,9 @@ $ curl 'http://localhost:8020/timeofday/v1/time?timezone=America/Los_Angeles' ``` And that's the end of Part 2. If you've gotten this far, -congratulations! On to [Part 3](TK:). +congratulations! Just a reminder, a working version of this server is +available under the "0.2.0" tag +[at the repo](https://github.com/elfsternberg/go-swagger-tutorial/tree/0.2.0). + +On to [Part 3](TK) diff --git a/docs/Part_03.md b/docs/Part_03.md index 31af0cd..d3d3d08 100644 --- a/docs/Part_03.md +++ b/docs/Part_03.md @@ -1,17 +1,18 @@ -# Adding Command Line Arguments to the Swagger Server +The first two parts of my swagger tutorial +[[Part 1](http://www.elfsternberg.com/2018/03/30/writing-microservice-swagger-part-1-specification/), +[Part 2](http://www.elfsternberg.com/2018/03/30/writing-microservice-swagger-part-2-business-logic/)] +were dedicated to the straightforward art of getting swagger up and +running. While I hope they're helpful, the whole point of those was to +get you to the point where you had the Timezone project, so I could show +you how to add Command Line Arguments to a Swagger microservice. -The first two parts of my swagger tutorial were dedicated to the -straightforward art of getting swagger up and running. While I hope -they're helpful, the whole point of those was to get you to the point -where you had the Timezone project, so I could teach how to extend the -Command Line Arguments of a swagger project. - -Nobody else, so far as I know, knows how to do this. One thing that I -emphasized in [Go Swagger Part 2](TK:) was that `configure_timeofday.go` -is the *only* file you should be touching, it's the interface between -the server and your business logic. Every example of adding new flags -to the command line, even the ones provided by the GoSwagger authors, -starts by modifying the file `cmd/\-server/main.go`, one of +One thing that I emphasized in +[Go Swagger Part 2](http://www.elfsternberg.com/2018/03/30/writing-microservice-swagger-part-2-business-logic/) +was that `configure_timeofday.go` is the *only* file you should be +touching, it's the interface between the server and your business logic. +Every example of adding new flags to the command line, even +[the one provided by the GoSwagger authors](https://github.com/go-openapi/kvstore/blob/master/cmd/kvstored/main.go#L50-L57), +starts by modifying the file `cmd/-server/main.go`, one of those files clearly marked `// DO NOT EDIT`. We're not going to edit files marked `// DO NOT EDIT`. @@ -23,7 +24,7 @@ uses for handling command line arguments, `go-flags`. `go-flags` is the tool Swagger uses by default for handling command line arguments. It's a clever tool that uses Go's -[tags and retrospection](TK:) features to encade the details of the CLI +[tags and reflection](https://golang.org/pkg/reflect/) features to encade the details of the CLI directly into a structure that will hold the options passed in on the command line. @@ -129,9 +130,17 @@ do we set the context? The *correct* way is to modify the handlers so they have the context when they're called upon. The way we do that is via the oldest object-oriented technique of all time, one that dates all the way back -to 1959 and the invention of Lisp: *closures*. +to 1964 and the invention of Lisp: +*[closures](https://tour.golang.org/moretypes/25)*. A closure *wraps* +one or more functions in an environment (a collection of variables +outside those functions), and preserves handles to those variables even +when those functions are passed out of the environment as references. A +garbage-collected language like Go makes this an especially powerful +technique because it means that anything in the environment for which +you *don't* keep handles will get collected, leaving only what matters. -Remember these lines in `configure_timeofday.go`, from way back? +So, let's do it. Remember these lines in `configure_timeofday.go`, from +way back? ``` api.TestGetHandler = operations.TestGetHandlerFunc(func(params operations.TestGetParams) middleware.Responder { @@ -151,24 +160,25 @@ replace those lines, *again*, so they look like this: Those are no longer references to functions. They're *function calls*! What do those functions return? Well, we know TimeGetHandlerFunc() is -expecting a function, so so that function call had better return a -function. +expecting a referece to a function, so so that function call had better +return a reference to a function. -And indeed it does. +And indeed it does: ``` func GetTime(timezone *Timezone) func(operations.TimeGetParams) middleware.Responder{ defaultTZ := timezone.Timezone - + + // Here's the function we return: return func(params operations.TimeGetParams) middleware.Responder { - // Everything else is the same... except we need *two* levels of - // closing } at the end! + // Everything else is the same... except we need *two* levels of + // closing } at the end! ``` Now, instead of returning a function defined at compile time, we -returned a function that is finalized when GetTime() is called, and it -now holds a permanent reference to our Timezone object. Do the same -thing for `PostTime`. +returned a function reference that is finalized when GetTime() is +called, and it now holds a permanent reference to our Timezone object. +Do the same thing for `PostTime`. There's one more thing we have to do. We've moved our default timezone to the `configure_timeofday.go` file, so we don't need it here anymore: @@ -190,8 +200,31 @@ And that's it. That's everything. You can add all the command line arguments you want, and only preserve the fields that are relevant to the particular handler you're going to invoke. +You can now build and run the server, but with a command line: + +``` +$ go build ./cmd/timeofday-server/ +$ ./timeofday-server --port=8080 --timezone="America/Los_Angeles" +``` + +And test it with curl: + +``` +$ curl 'http://localhost:8020/timeofday/v1/time?timezone=America/New_York' +{"timeofday":"2018-03-30 23:44:47.701895604 -0400 EDT"} +$ curl 'http://localhost:8020/timeofday/v1/time' +{"timeofday":"2018-03-30 20:44:54.525313806 -0700 PDT"} +``` + +Note that the default timezone is now PDT, or Pacific Daily Time, which +corresponds to the America/Los_Angeles entry in the database in late +March. + And *that's* how you add command line arguments to Swagger servers -correctly. +correctly without exposing your CLI settings to every other function in +your server. If you want to see the entirely of the source code, the +[advanced version on the repository](https://github.com/elfsternberg/go-swagger-tutorial/tree/0.4.0) +has it all. From a83e006f8d1694118920b631be47e6d079c2164f Mon Sep 17 00:00:00 2001 From: "Elf M. Sternberg" Date: Fri, 30 Mar 2018 22:26:31 -0700 Subject: [PATCH 3/3] Documentation done, and Readme annotated with the appropriate links. --- README.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 314124c..0e0627f 100644 --- a/README.md +++ b/README.md @@ -6,8 +6,10 @@ single parameter either via GET or POST to specify the timezone from which the client wants the time. This repository exists as a supplement to my tutorial, -[Adding Command Line Arguments to Go-Swagger](TK:), in which I do -exactly that. +[Adding Command Line Arguments to Go Swagger Microservices](http://www.elfsternberg.com/2018/03/30/writing-microservice-swagger-part-3-adding-command-line-arguments/), +in which I show how to do exactly that, by providing a dynamic way to +configure the default timezone at server start-up, via the CLI or an +environment variable. # Status