Skip to content

Commit

Permalink
feat: adds support for OAS 3.1 (#379)
Browse files Browse the repository at this point in the history
* chore: allow adding new test cases

* feat: adds support for OAS 3.1 #369

* chore: resolve merge conflicts

* chore: resolve linting errors

* chore: updates copyright

* chore: fix go mod errors

* feat: updates docs to add support for OAS 3.1 #369

* bug: set env before deployment #369

* bug: fixes error msg #369

* bug: fixes basepath override #369

* chore: add oas 3.1 test case #369

* chore: check oas version to skip oas policy #369

* chore: fix linting issue #369
  • Loading branch information
srinandan authored Jan 24, 2024
1 parent b20e621 commit fe41918
Show file tree
Hide file tree
Showing 13 changed files with 2,033 additions and 10 deletions.
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -26,3 +26,7 @@ remote-service.*

#json files
*.json

!test/*.json
!test/*.yaml
!test/*.zip
9 changes: 6 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -128,7 +128,7 @@ docker run -ti ghcr.io/apigee/apigeecli:latest orgs list -t $token

### Using apigeecli with Cloud Build

To execute apigeecli commands in cloud build,
To execute apigeecli commands in cloud build,

```
steps:
Expand All @@ -143,6 +143,7 @@ steps:
```

If you need the response from the previous command as input to the next, then take advantage of `sh` and `jq` like so:

```yaml
steps:
- id: 'Run apigeecli commands'
Expand Down Expand Up @@ -194,19 +195,21 @@ The following environment variables may be set to control the behavior of `apige

`apigeecli` can generate API proxies from:

* OpenAPI 3.0 Specification
* OpenAPI 3.0/3.1 Specification
* GraphQL Schema
* A template/stub for Application Integration
* Cloud Endpoints/API Gateway OpenAPI 2.0 specification

### Generating API Proxies from OpenAPI Specs

`apigeecli` allows the user to generate Apigee API Proxy bundles from an OpenAPI spec (only 3.0.x supported). The Apigee control plane does not support custom formats (ex: uuid). If you spec contains custom formats, consider the following flags
`apigeecli` allows the user to generate Apigee API Proxy bundles from an OpenAPI spec (3.0.x and 3.1.x are supported). The Apigee control plane does not support custom formats (ex: uuid). If you spec contains custom formats, consider the following flags

* `--add-cors=true`: Add a CORS policy
* `--formatValidation=false`: this disables validation for custom formats.
* `--skip-policy=false`: By default the OAS policy is added to the proxy (to validate API requests). By setting this to false, schema validation is not enabled and the control plane will not reject the bundle due to custom formats.

**NOTE**: The Apigee runtime does not support OAS 3.1.x (specifically the OpenAPI Validation policy). When using OAS 3.1.x, `apigeecli` generates the proxy, but does not include the OAS validation policy.

The following actions are automatically implemented when the API Proxy bundle is generated:

#### Security Policies
Expand Down
3 changes: 1 addition & 2 deletions cmd/apis/bundlecrtapis.go
Original file line number Diff line number Diff line change
Expand Up @@ -118,8 +118,7 @@ func init() {
BundleCreateCmd.Flags().BoolVarP(&sequencedRollout, "sequencedrollout", "",
false, "If set to true, the routing rules will be rolled out in a safe order; default is false")
BundleCreateCmd.Flags().BoolVarP(&safeDeploy, "safedeploy", "",
true, "When set to true, generateDeployChangeReport will be executed and "+
"deployment will proceed if there are no conflicts; default is true")
true, deploymentMsg)
BundleCreateCmd.Flags().StringVarP(&serviceAccountName, "sa", "s",
"", "The format must be {ACCOUNT_ID}@{PROJECT}.iam.gserviceaccount.com.")

Expand Down
3 changes: 3 additions & 0 deletions cmd/apis/common.go
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,9 @@ import (
"internal/clilog"
)

const deploymentMsg = "When set to true, generateDeployChangeReport will be executed and " +
"deployment will proceed if there are no conflicts; default is true"

func GetRevision(respBody []byte) (revision int, err error) {
var apiProxyRevResp map[string]interface{}

Expand Down
4 changes: 3 additions & 1 deletion cmd/apis/crtapi.go
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,9 @@ var (
)

func init() {
CreateCmd.AddCommand(OasCreateCmd)
// disable v1 of OasCreate
// CreateCmd.AddCommand(OasCreateCmd)
CreateCmd.AddCommand(OasCreatev2Cmd)
CreateCmd.AddCommand(GhCreateCmd)
CreateCmd.AddCommand(BundleCreateCmd)
CreateCmd.AddCommand(GqlCreateCmd)
Expand Down
3 changes: 1 addition & 2 deletions cmd/apis/depapi.go
Original file line number Diff line number Diff line change
Expand Up @@ -76,8 +76,7 @@ func init() {
DepCmd.Flags().BoolVarP(&sequencedRollout, "sequencedrollout", "",
false, "If set to true, the routing rules will be rolled out in a safe order; default is false")
DepCmd.Flags().BoolVarP(&safeDeploy, "safedeploy", "",
true, "When set to true, generateDeployChangeReport will be executed and "+
"deployment will proceed if there are no conflicts; default is true")
true, deploymentMsg)
DepCmd.Flags().StringVarP(&serviceAccountName, "sa", "s",
"", "The format must be {ACCOUNT_ID}@{PROJECT}.iam.gserviceaccount.com.")

Expand Down
225 changes: 225 additions & 0 deletions cmd/apis/oascrtapisv2.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,225 @@
// Copyright 2023 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package apis

import (
"fmt"
"os"
"regexp"

"internal/apiclient"
"internal/clilog"

bundle "internal/bundlegen"
proxybundle "internal/bundlegen/proxybundle"

"internal/client/apis"

"github.com/spf13/cobra"
)

var OasCreatev2Cmd = &cobra.Command{
Use: "openapi",
Aliases: []string{"oas"},
Short: "Creates an API proxy from an OpenAPI Specification",
Long: "Creates an API proxy from an OpenAPI Specification",
Args: func(cmd *cobra.Command, args []string) (err error) {
if oasFile == "" && oasURI == "" {
return fmt.Errorf("either oas-base-folderpath or oas-base-uri must be passed")
}
if targetURL != "" && targetURLRef != "" {
return fmt.Errorf("either target-url or target-url-ref must be passed, not both")
}
if integration != "" && apitrigger == "" {
return fmt.Errorf("apitrigger must be passed if integration is set")
}
if integration == "" && apitrigger != "" {
return fmt.Errorf("integration must be passed if apitrigger is set")
}
if (targetURL != "" || targetURLRef != "") && (integration != "" || apitrigger != "") {
return fmt.Errorf("integration or apitrigger cannot be set if targetURL or targetURLRef is set")
}
if env != "" {
apiclient.SetApigeeEnv(env)
}
apiclient.SetRegion(region)
return apiclient.SetApigeeOrg(org)
},
RunE: func(cmd *cobra.Command, args []string) (err error) {
var content []byte

integrationEndpoint := false

if oasFile != "" {
err = checkFolder()
if err != nil {
return err
}
}

content, err = bundle.LoadDocument(oasFile, oasURI, specName, validateSpec)
if err != nil {
return err
}

version := bundle.GetModelVersion()
if version != "" {
re := regexp.MustCompile(`3\.1\.[0-9]`)
if re.MatchString(version) {
clilog.Warning.Println("OpenAPI 3.1 detected. Skipping policy validation.")
skipPolicy = true
}
}

targetOptions := bundle.TargetOptions{
IntegrationBackend: bundle.IntegrationBackendOptions{
IntegrationName: integration,
TriggerName: apitrigger,
},
HttpBackend: bundle.HttpBackendOptions{
OasGoogleAcessTokenScopeLiteral: oasGoogleAcessTokenScopeLiteral,
OasGoogleIDTokenAudLiteral: oasGoogleIDTokenAudLiteral,
OasGoogleIDTokenAudRef: oasGoogleIDTokenAudRef,
OasTargetURLRef: targetURLRef,
TargetURL: targetURL,
},
}

// check if integrationEndpoint is selected
if integration != "" {
integrationEndpoint = true
}

// Generate the apiproxy struct
err = bundle.GenerateAPIProxyDefFromOASv2(name,
basePath,
specName,
skipPolicy,
addCORS,
integrationEndpoint,
oasGoogleAcessTokenScopeLiteral,
oasGoogleIDTokenAudLiteral,
oasGoogleIDTokenAudRef,
targetURLRef,
targetURL)

if err != nil {
return err
}

// Create the API proxy bundle
err = proxybundle.GenerateAPIProxyBundleFromOAS(name,
string(content),
specName,
skipPolicy,
addCORS,
targetOptions)

if err != nil {
return err
}

if importProxy {
respBody, err := apis.CreateProxy(name, name+zipExt)
if err != nil {
return err
}
if env != "" {
clilog.Info.Printf("Deploying the API Proxy %s to environment %s\n", name, env)
if revision, err = GetRevision(respBody); err != nil {
return err
}
if _, err = apis.DeployProxy(name, revision, overrides,
sequencedRollout, safeDeploy, serviceAccountName); err != nil {
return err
}
if wait {
return Wait(name, revision)
}
}
}

return err
},
}

var specName string

func init() {
OasCreatev2Cmd.Flags().StringVarP(&name, "name", "n",
"", "API Proxy name")
OasCreatev2Cmd.Flags().StringVarP(&basePath, "basepath", "p",
"", "Base Path of the API Proxy; Overrides the basePath in spec")
OasCreatev2Cmd.Flags().StringVarP(&oasFile, "oas-base-folderpath", "f",
"", "Open API Spec Folder")
OasCreatev2Cmd.Flags().StringVarP(&oasURI, "oas-base-uri", "u",
"", "Open API Specification URI Base location")
OasCreatev2Cmd.Flags().StringVarP(&specName, "oas-name", "",
"", "Open API 3.0/3.1 Specification Name; Used with oas-base-filepath or oas-base-uri")
OasCreatev2Cmd.Flags().StringVarP(&oasGoogleAcessTokenScopeLiteral, "google-accesstoken-scope-literal", "",
"", "Generate Google Access token with target endpoint and set scope")
OasCreatev2Cmd.Flags().StringVarP(&oasGoogleIDTokenAudLiteral, "google-idtoken-aud-literal", "",
"", "Generate Google ID token with target endpoint and set audience")
OasCreatev2Cmd.Flags().StringVarP(&oasGoogleIDTokenAudRef, "google-idtoken-aud-ref", "",
"", "Generate Google ID token token with target endpoint and set audience reference")
OasCreatev2Cmd.Flags().StringVarP(&targetURLRef, "target-url-ref", "",
"", "Set a reference variable containing the target endpoint")
OasCreatev2Cmd.Flags().StringVarP(&targetURL, "target-url", "",
"", "Set a target URL for the target endpoint")
OasCreatev2Cmd.Flags().StringVarP(&integration, "integration", "i",
"", "Integration name")
OasCreatev2Cmd.Flags().StringVarP(&apitrigger, "trigger", "",
"", "API Trigger name; don't include 'api_trigger/'")
OasCreatev2Cmd.Flags().BoolVarP(&importProxy, "import", "",
true, "Import API Proxy after generation from spec")
OasCreatev2Cmd.Flags().BoolVarP(&validateSpec, "validate", "",
true, "Validate Spec before generating proxy")
OasCreatev2Cmd.Flags().BoolVarP(&skipPolicy, "skip-policy", "",
false, "Skip adding the OAS Validate policy")
OasCreatev2Cmd.Flags().BoolVarP(&addCORS, "add-cors", "",
false, "Add a CORS policy")

OasCreatev2Cmd.Flags().StringVarP(&env, "env", "e",
"", "Apigee environment name")
OasCreatev2Cmd.Flags().BoolVarP(&overrides, "ovr", "",
false, "Forces deployment of the new revision")
OasCreatev2Cmd.Flags().BoolVarP(&wait, "wait", "",
false, "Waits for the deployment to finish, with success or error")
OasCreatev2Cmd.Flags().BoolVarP(&sequencedRollout, "sequencedrollout", "",
false, "If set to true, the routing rules will be rolled out in a safe order; default is false")
OasCreatev2Cmd.Flags().BoolVarP(&safeDeploy, "safedeploy", "",
true, deploymentMsg)
OasCreatev2Cmd.Flags().StringVarP(&serviceAccountName, "sa", "s",
"", "The format must be {ACCOUNT_ID}@{PROJECT}.iam.gserviceaccount.com.")

_ = OasCreatev2Cmd.MarkFlagRequired("name")
_ = OasCreatev2Cmd.MarkFlagRequired("oas-name")
}

func checkFolder() error {
f, err := os.Open(oasFile)
if err != nil {
return err
}
defer f.Close()
fInfo, err := f.Stat()
if err != nil {
return err
}
if !fInfo.IsDir() {
return fmt.Errorf("oas-base-folderpath must be a folder")
}
return nil
}
3 changes: 1 addition & 2 deletions cmd/apis/undepapi.go
Original file line number Diff line number Diff line change
Expand Up @@ -53,8 +53,7 @@ func init() {
UndepCmd.Flags().IntVarP(&revision, "rev", "v",
-1, "API Proxy revision")
UndepCmd.Flags().BoolVarP(&safeUndeploy, "safeundeploy", "",
true, "When set to true, generateUndeployChangeReport will be executed and "+
"undeployment will proceed if there are no conflicts; default is true")
true, deploymentMsg)

_ = UndepCmd.MarkFlagRequired("env")
_ = UndepCmd.MarkFlagRequired("name")
Expand Down
10 changes: 10 additions & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,11 @@ require github.com/spf13/cobra v1.6.1

require (
cloud.google.com/go/compute/metadata v0.2.0 // indirect
github.com/bahlo/generic-list-go v0.2.0 // indirect
github.com/buger/jsonparser v1.1.1 // indirect
github.com/cpuguy83/go-md2man/v2 v2.0.2 // indirect
github.com/decred/dcrd/dcrec/secp256k1/v4 v4.2.0 // indirect
github.com/dprotaso/go-yit v0.0.0-20220510233725-9ba8df137936 // indirect
github.com/getkin/kin-openapi v0.115.0 // indirect
github.com/ghodss/yaml v1.0.0 // indirect
github.com/go-openapi/jsonpointer v0.19.6 // indirect
Expand All @@ -43,14 +46,21 @@ require (
github.com/lestrrat-go/option v1.0.1 // indirect
github.com/mailru/easyjson v0.7.7 // indirect
github.com/mohae/deepcopy v0.0.0-20170929034955-c48cc78d4826 // indirect
github.com/pb33f/libopenapi v0.15.1 // indirect
github.com/pb33f/libopenapi-validator v0.0.40 // indirect
github.com/perimeterx/marshmallow v1.1.4 // indirect
github.com/russross/blackfriday/v2 v2.1.0 // indirect
github.com/santhosh-tekuri/jsonschema/v5 v5.3.1 // indirect
github.com/segmentio/asm v1.2.0 // indirect
github.com/spf13/pflag v1.0.5 // indirect
github.com/thedevsaddam/gojsonq v2.3.0+incompatible // indirect
github.com/vmware-labs/yaml-jsonpath v0.3.2 // indirect
github.com/wk8/go-ordered-map/v2 v2.1.8 // indirect
golang.org/x/crypto v0.17.0 // indirect
golang.org/x/exp v0.0.0-20230811145659-89c5cff77bcb // indirect
golang.org/x/net v0.17.0 // indirect
golang.org/x/oauth2 v0.6.0 // indirect
golang.org/x/sync v0.1.0 // indirect
golang.org/x/sys v0.15.0 // indirect
golang.org/x/time v0.3.0 // indirect
google.golang.org/appengine v1.6.7 // indirect
Expand Down
Loading

0 comments on commit fe41918

Please sign in to comment.