Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: added mirroring support #327

Open
wants to merge 13 commits into
base: main
Choose a base branch
from
16 changes: 14 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ The Event Streams service supports payload data encryption that uses a root key
* [Basic example](./examples/basic)
* [Complete example with topics and schema creation.](./examples/complete)
* [Financial Services Cloud profile example](./examples/fscloud)
* [Mirroring example](./examples/mirroring)
* [Contributing](#contributing)
<!-- END OVERVIEW HOOK -->

Expand Down Expand Up @@ -106,26 +107,32 @@ You need the following permissions to run this module.
| Name | Version |
|------|---------|
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 1.3.0 |
| <a name="requirement_ibm"></a> [ibm](#requirement\_ibm) | >= 1.70.0, <2.0.0 |
| <a name="requirement_ibm"></a> [ibm](#requirement\_ibm) | >= 1.71.0, <2.0.0 |
| <a name="requirement_time"></a> [time](#requirement\_time) | >= 0.9.1 |

### Modules

| Name | Source | Version |
|------|--------|---------|
| <a name="module_cbr_rule"></a> [cbr\_rule](#module\_cbr\_rule) | terraform-ibm-modules/cbr/ibm//modules/cbr-rule-module | 1.28.1 |
| <a name="module_es_guid_crn_parser"></a> [es\_guid\_crn\_parser](#module\_es\_guid\_crn\_parser) | terraform-ibm-modules/common-utilities/ibm//modules/crn-parser | 1.1.0 |

### Resources

| Name | Type |
|------|------|
| [ibm_event_streams_mirroring_config.es_mirroring_config](https://registry.terraform.io/providers/IBM-Cloud/ibm/latest/docs/resources/event_streams_mirroring_config) | resource |
| [ibm_event_streams_schema.es_schema](https://registry.terraform.io/providers/IBM-Cloud/ibm/latest/docs/resources/event_streams_schema) | resource |
| [ibm_event_streams_topic.es_topic](https://registry.terraform.io/providers/IBM-Cloud/ibm/latest/docs/resources/event_streams_topic) | resource |
| [ibm_iam_authorization_policy.en_service_policy](https://registry.terraform.io/providers/IBM-Cloud/ibm/latest/docs/resources/iam_authorization_policy) | resource |
| [ibm_iam_authorization_policy.kms_policy](https://registry.terraform.io/providers/IBM-Cloud/ibm/latest/docs/resources/iam_authorization_policy) | resource |
| [ibm_iam_authorization_policy.mirroring_policy](https://registry.terraform.io/providers/IBM-Cloud/ibm/latest/docs/resources/iam_authorization_policy) | resource |
| [ibm_resource_instance.es_instance](https://registry.terraform.io/providers/IBM-Cloud/ibm/latest/docs/resources/resource_instance) | resource |
| [ibm_resource_key.service_credentials](https://registry.terraform.io/providers/IBM-Cloud/ibm/latest/docs/resources/resource_key) | resource |
| [ibm_resource_tag.es_access_tag](https://registry.terraform.io/providers/IBM-Cloud/ibm/latest/docs/resources/resource_tag) | resource |
| [time_sleep.wait_for_authorization_policy](https://registry.terraform.io/providers/hashicorp/time/latest/docs/resources/sleep) | resource |
| [time_sleep.wait_for_en_service_policy](https://registry.terraform.io/providers/hashicorp/time/latest/docs/resources/sleep) | resource |
| [time_sleep.wait_for_kms_authorization_policy](https://registry.terraform.io/providers/hashicorp/time/latest/docs/resources/sleep) | resource |
| [time_sleep.wait_for_mirroring_authorization_policy](https://registry.terraform.io/providers/hashicorp/time/latest/docs/resources/sleep) | resource |

### Inputs

Expand All @@ -140,6 +147,9 @@ You need the following permissions to run this module.
| <a name="input_kms_encryption_enabled"></a> [kms\_encryption\_enabled](#input\_kms\_encryption\_enabled) | Set this to true to control the encryption keys used to encrypt the data that you store in IBM Cloud® Databases. If set to false, the data is encrypted by using randomly generated keys. For more info on Key Protect integration, see https://cloud.ibm.com/docs/cloud-databases?topic=cloud-databases-key-protect. For more info on HPCS integration, see https://cloud.ibm.com/docs/cloud-databases?topic=cloud-databases-hpcs | `bool` | `false` | no |
| <a name="input_kms_key_crn"></a> [kms\_key\_crn](#input\_kms\_key\_crn) | The root key CRN of the key management service (Key Protect or Hyper Protect Crypto Services) to use to encrypt the payload data. [Learn more](https://cloud.ibm.com/docs/EventStreams?topic=EventStreams-managing_encryption) about integrating Hyper Protect Crypto Services with Event Streams. | `string` | `null` | no |
| <a name="input_metrics"></a> [metrics](#input\_metrics) | Enhanced metrics to activate, as list of strings. Only allowed for enterprise plans. Allowed values: 'topic', 'partition', 'consumers'. | `list(string)` | `[]` | no |
| <a name="input_mirroring"></a> [mirroring](#input\_mirroring) | Mirroring configuration | <pre>object({<br/> source_crn = string<br/> source_alias = string<br/> target_alias = string<br/> })</pre> | `null` | no |
| <a name="input_mirroring_enabled"></a> [mirroring\_enabled](#input\_mirroring\_enabled) | Set this to true to enable mirroring. Mirroring enables messages in one Event Streams service instance to be continuously copied to a second instance to increase resiliency. See https://cloud.ibm.com/docs/EventStreams?topic=EventStreams-mirroring. | `bool` | `false` | no |
| <a name="input_mirroring_topic_patterns"></a> [mirroring\_topic\_patterns](#input\_mirroring\_topic\_patterns) | The list of the topics to set in instance. Required only if var.mirroring\_enabled is set to true. | `list(string)` | `null` | no |
| <a name="input_plan"></a> [plan](#input\_plan) | The plan for the Event Streams instance. Possible values: `lite`, `standard`, `enterprise-3nodes-2tb`. | `string` | `"standard"` | no |
| <a name="input_region"></a> [region](#input\_region) | The region where the Event Streams are created. | `string` | `"us-south"` | no |
| <a name="input_resource_group_id"></a> [resource\_group\_id](#input\_resource\_group\_id) | The resource group ID where the Event Streams instance is created. | `string` | n/a | yes |
Expand All @@ -163,6 +173,8 @@ You need the following permissions to run this module.
| <a name="output_kafka_broker_version"></a> [kafka\_broker\_version](#output\_kafka\_broker\_version) | The Kafka version |
| <a name="output_kafka_brokers_sasl"></a> [kafka\_brokers\_sasl](#output\_kafka\_brokers\_sasl) | (Array of Strings) Kafka brokers use for interacting with Kafka native API |
| <a name="output_kafka_http_url"></a> [kafka\_http\_url](#output\_kafka\_http\_url) | The API endpoint to interact with Event Streams REST API |
| <a name="output_mirroring_config_id"></a> [mirroring\_config\_id](#output\_mirroring\_config\_id) | The ID of the mirroring config in CRN format |
| <a name="output_mirroring_topic_patterns"></a> [mirroring\_topic\_patterns](#output\_mirroring\_topic\_patterns) | Mirroring topic patterns |
| <a name="output_service_credentials_json"></a> [service\_credentials\_json](#output\_service\_credentials\_json) | The service credentials JSON map. |
| <a name="output_service_credentials_object"></a> [service\_credentials\_object](#output\_service\_credentials\_object) | The service credentials object. |
<!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
Expand Down
2 changes: 1 addition & 1 deletion examples/basic/version.tf
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ terraform {
required_providers {
ibm = {
source = "IBM-Cloud/ibm"
version = "= 1.70.0"
version = "= 1.71.0"
}
}
}
4 changes: 2 additions & 2 deletions examples/complete/variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -13,13 +13,13 @@ variable "region" {
variable "prefix" {
type = string
description = "The prefix to apply to all resources created by this example."
default = "event-streams-com"
default = "geretain-sky"
Ak-sky marked this conversation as resolved.
Show resolved Hide resolved
}

variable "resource_group" {
type = string
description = "An existing resource group name to use for this example. If not specified, a new resource group is created."
default = null
default = "geretain-aakash-rg"
Ak-sky marked this conversation as resolved.
Show resolved Hide resolved
}

variable "resource_tags" {
Expand Down
2 changes: 1 addition & 1 deletion examples/complete/version.tf
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ terraform {
# Use latest version of provider in non-basic examples to verify latest version works with module
ibm = {
source = "IBM-Cloud/ibm"
version = ">= 1.70.0"
version = ">= 1.71.0"
}
}
}
10 changes: 10 additions & 0 deletions examples/fscloud/outputs.tf
Original file line number Diff line number Diff line change
Expand Up @@ -43,3 +43,13 @@ output "service_credentials_object" {
value = module.event_streams.service_credentials_object
sensitive = true
}

output "mirroring_config_id" {
description = "The ID of the mirroring config in CRN format"
value = module.event_streams.mirroring_config_id
}

output "mirroring_topic_patterns" {
description = "Mirroring topic patterns"
value = module.event_streams.mirroring_topic_patterns
}
2 changes: 1 addition & 1 deletion examples/fscloud/version.tf
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ terraform {
# Use latest version of provider in non-basic examples to verify latest version works with module
ibm = {
source = "IBM-Cloud/ibm"
version = ">= 1.70.1"
version = ">= 1.71.0"
}
}
}
9 changes: 9 additions & 0 deletions examples/mirroring/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# Mirroring example

An end-to-end example that creates an IBM Event Streams for IBM Cloud instance and a mirroring instance.

This example uses the IBM Cloud Terraform provider to create the following infrastructure.

- A new resource group, if one is not passed in.
- A instance of Event Streams in the provided resource group and region.
- A instance of Event Streams in the provided resource group and region with mirroring enabled.
42 changes: 42 additions & 0 deletions examples/mirroring/main.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
##############################################################################
# Resource Group
##############################################################################

module "resource_group" {
source = "terraform-ibm-modules/resource-group/ibm"
version = "1.1.6"
# if an existing resource group is not set (null) create a new one using prefix
resource_group_name = var.resource_group == null ? "${var.prefix}-resource-group" : null
existing_resource_group_name = var.resource_group
}

##############################################################################
# Events-streams-source-instance
##############################################################################
module "source_event_streams" {
source = "../../"
resource_group_id = module.resource_group.resource_group_id
es_name = "${var.prefix}-es"
tags = var.resource_tags
plan = "enterprise-3nodes-2tb"
}

##############################################################################
# Events-streams-mirroring-instance
##############################################################################

module "event_streams_mirror" {
depends_on = [module.source_event_streams]
source = "../../"
resource_group_id = module.resource_group.resource_group_id
es_name = "${var.prefix}-mirror"
tags = var.resource_tags
plan = "enterprise-3nodes-2tb"
mirroring_enabled = true
mirroring_topic_patterns = ["topic-1", "topic-2"]
mirroring = {
source_crn = module.source_event_streams.crn
source_alias = "source-alias"
target_alias = "target-alias"
}
}
48 changes: 48 additions & 0 deletions examples/mirroring/outputs.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
##############################################################################
# Outputs
##############################################################################

output "resource_group_name" {
description = "Resource group name"
value = module.resource_group.resource_group_name
}

output "resource_group_id" {
description = "Resource group ID"
value = module.resource_group.resource_group_id
}

output "crn" {
description = "Event Streams mirror instance crn"
value = module.event_streams_mirror.crn
}

output "guid" {
description = "Event Streams instance guid"
value = module.event_streams_mirror.guid
}

output "kafka_brokers_sasl" {
description = "(Array of Strings) Kafka brokers use for interacting with Kafka native API"
value = module.event_streams_mirror.kafka_brokers_sasl
}

output "kafka_http_url" {
description = "The API endpoint to interact with Event Streams REST API"
value = module.event_streams_mirror.kafka_http_url
}

output "kafka_broker_version" {
description = "The Kafka version"
value = module.event_streams_mirror.kafka_broker_version
}

output "mirroring_config_id" {
description = "The ID of the mirroring config in CRN format"
value = module.event_streams_mirror.mirroring_config_id
}

output "mirroring_topic_patterns" {
description = "Mirroring topic patterns"
value = module.event_streams_mirror.mirroring_topic_patterns
}
4 changes: 4 additions & 0 deletions examples/mirroring/provider.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
provider "ibm" {
ibmcloud_api_key = var.ibmcloud_api_key
region = var.region
}
29 changes: 29 additions & 0 deletions examples/mirroring/variables.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
variable "ibmcloud_api_key" {
type = string
description = "The IBM Cloud API key."
sensitive = true
}

variable "region" {
type = string
description = "The region where the Event Streams mirroring instance is created."
default = "us-south"
}

variable "prefix" {
type = string
description = "The prefix to apply to all resources created by this example."
default = "event-streams"
}

variable "resource_group" {
type = string
description = "An existing resource group name to use for this example. If not specified, a new resource group is created."
default = null
}

variable "resource_tags" {
type = list(string)
description = "The list of tags associated with the Event Steams instance."
default = []
}
9 changes: 9 additions & 0 deletions examples/mirroring/version.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
terraform {
required_version = ">= 1.3.0"
required_providers {
ibm = {
source = "IBM-Cloud/ibm"
version = ">= 1.71.0"
}
}
}
Loading