Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: update mirroring example doc #5767

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
108 changes: 107 additions & 1 deletion examples/ibm-event-streams/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ The following `"ibm_resource_instance"` arguments are required:

- `resource_group_id`: The ID of the resource group in which the instance will be provisioned. For more information about resource groups, see [Managing resource groups](https://cloud.ibm.com/docs/account?topic=account-rgs).

The `parameters` argument is optional and provides additional provision or update options. Supported parameters are:
The `parameters/parameters_json` argument is optional and provides additional provision or update options. Supported parameters are:

- `throughput`: One of `"150"` (the default), `"300"`, `"450"`. The maximum capacity in MB/s for producing or consuming messages. For more information see [Scaling Enterprise plan capacity](https://cloud.ibm.com/docs/EventStreams?topic=EventStreams-ES_scaling_capacity). *Note:* See [Scaling combinations](https://cloud.ibm.com/docs/EventStreams?topic=EventStreams-ES_scaling_capacity#ES_scaling_combinations) for allowed combinations of `throughput` and `storage_size`.
- Example: `throughput = "300"`
Expand All @@ -38,6 +38,41 @@ The `parameters` argument is optional and provides additional provision or updat
- `kms_key_crn`: The CRN (as a string) of a customer-managed root key provisioned with an IBM Cloud Key Protect or Hyper Protect Crypto Service. If provided, this key is used to encrypt all data at rest. For more information on customer-managed encryption, see [Managing encryption in Event Streams](https://cloud.ibm.com/docs/EventStreams?topic=EventStreams-managing_encryption).
- Example: `kms_key_crn = "crn:v1:prod:public:kms:us-south:a/6db1b0d0b5c54ee5c201552547febcd8:20adf7eb-e095-4dec-08cf-0b7d81e32db6:key:3fa9d921-d3b6-3516-a1ec-d54e27e7638b"`

- `mirroring`: To enable mirroring in the cluster using `parameters_json`. For enterprise instance only. If defined `source_crn` (source cluster CRN as a string), `source_alias` (alias for source cluster as a string), `target_alias` (alias for target cluster as a string) are required. `options` are optional.
- Example:

```terraform
parameters_json = jsonencode(
{
mirroring = {
source_crn = data.ibm_resource_instance.es_instance_source.id
source_alias = "source-alias"
target_alias = "target-alias"
options = {
topic_name_transform = {
type = "rename"
rename = {
add_prefix = "add_prefix"
add_suffix = "add_suffix"
remove_prefix = "remove_prefix"
remove_suffix = "remove_suffix"
}
}
group_id_transform = {
type = "rename"
rename = {
add_prefix = "add_prefix"
add_suffix = "add_suffix"
remove_prefix = "remove_prefix"
remove_suffix = "remove_suffix"
}
}
}
}
}
)
```

The `timeouts` argument is used to specify how long the IBM Cloud terraform provider will wait for the provision, update, or deprovision of the service instance. Values of 15 minutes are sufficient for standard and lite plans. For enterprise plans:
- Use "3h" for create. Add an additional 1 hour for each level of non-default throughput, and an additional 30 minutes for each level of non-default storage size. For example with `throughput = "300"` (one level over default) and `storage_size = "8192"` (three levels over default), use 3 hours + 1 * 1 hour + 3 * 30 minutes = 5.5 hours.
- Use "1h" for update. If increasing the throughput or storage size, add an additional 1 hour for each level of non-default throughput, and an additional 30 minutes for each level of non-default storage size.
Expand Down Expand Up @@ -222,6 +257,77 @@ resource "kafka_consumer_app" "es_kafka_app" {
topics = [data.ibm_event_streams_topic.es_topic_7.name]
}
```
#### Scenario 8: Create a target Event Streams service instance with mirroring enabled and its mirroring config
```terraform
data "ibm_resource_instance" "es_instance_source" {
name = "terraform-integration-source"
resource_group_id = data.ibm_resource_group.group.id
}
# setup s2s at service level for mirroring to work
resource "ibm_iam_authorization_policy" "service-policy" {
source_service_name = "messagehub"
target_service_name = "messagehub"
roles = ["Reader"]
description = "test mirroring setup via terraform"
}

resource "ibm_resource_instance" "es_instance_target" {
name = "terraform-integration-target"
service = "messagehub"
plan = "enterprise-3nodes-2tb"
location = "us-south"
resource_group_id = data.ibm_resource_group.group.id
parameters_json = jsonencode(
{
mirroring = {
source_crn = data.ibm_resource_instance.es_instance_source.id
source_alias = "source-alias"
target_alias = "target-alias"
options = {
topic_name_transform = {
type = "rename"
rename = {
add_prefix = "add_prefix"
add_suffix = "add_suffix"
remove_prefix = "remove_prefix"
remove_suffix = "remove_suffix"
}
}
group_id_transform = {
type = "rename"
rename = {
add_prefix = "add_prefix"
add_suffix = "add_suffix"
remove_prefix = "remove_prefix"
remove_suffix = "remove_suffix"
}
}
}
}
}
)
timeouts {
create = "3h"
update = "1h"
delete = "15m"
}
}
# Configure a service-to-service binding between both instances to allow both instances to communicate.
resource "ibm_iam_authorization_policy" "instance_policy" {
source_service_name = "messagehub"
source_resource_instance_id = ibm_resource_instance.es_instance_target.guid
target_service_name = "messagehub"
target_resource_instance_id = data.ibm_resource_instance.es_instance_source.guid
roles = ["Reader"]
description = "test mirroring setup via terraform"
}

# Select some topics from the source cluster to mirror.
resource "ibm_event_streams_mirroring_config" "es_mirroring_config" {
resource_instance_id = ibm_resource_instance.es_instance_target.id
mirroring_topic_patterns = ["topicA", "topicB"]
}
```

## Dependencies

Expand Down
48 changes: 34 additions & 14 deletions examples/ibm-event-streams/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -31,17 +31,17 @@ resource "ibm_resource_instance" "es_instance_2" {
resource_group_id = data.ibm_resource_group.group.id

parameters = {
throughput = "300"
storage_size = "4096"
service-endpoints = "private"
private_ip_allowlist = "[10.0.0.0/32,10.0.0.1/32]"
metrics = "[topic,consumers]"
throughput = "300"
storage_size = "4096"
service-endpoints = "private"
private_ip_allowlist = "[10.0.0.0/32,10.0.0.1/32]"
metrics = "[topic,consumers]"
}

timeouts {
create = "330m" # 5.5h
update = "210m" # 3.5h
delete = "1h"
create = "330m" # 5.5h
update = "210m" # 3.5h
delete = "1h"
}
}

Expand Down Expand Up @@ -74,8 +74,8 @@ data "ibm_resource_instance" "es_instance_4" {

resource "ibm_event_streams_schema" "es_schema" {
resource_instance_id = data.ibm_resource_instance.es_instance_4.id
schema_id = "tf_schema"
schema = <<SCHEMA
schema_id = "tf_schema"
schema = <<SCHEMA
{
"type": "record",
"name": "record_name",
Expand Down Expand Up @@ -107,10 +107,10 @@ data "ibm_resource_instance" "es_instance_source" {
}
# setup s2s at service level for mirroring to work
resource "ibm_iam_authorization_policy" "service-policy" {
source_service_name = "messagehub"
target_service_name = "messagehub"
roles = ["Reader"]
description = "test mirroring setup via terraform"
source_service_name = "messagehub"
target_service_name = "messagehub"
roles = ["Reader"]
description = "test mirroring setup via terraform"
}

resource "ibm_resource_instance" "es_instance_target" {
Expand All @@ -125,6 +125,26 @@ resource "ibm_resource_instance" "es_instance_target" {
source_crn = data.ibm_resource_instance.es_instance_source.id
source_alias = "source-alias"
target_alias = "target-alias"
options = {
topic_name_transform = {
type = "rename"
rename = {
add_prefix = "add_prefix"
add_suffix = "add_suffix"
remove_prefix = "remove_prefix"
remove_suffix = "remove_suffix"
}
}
group_id_transform = {
type = "rename"
rename = {
add_prefix = "add_prefix"
add_suffix = "add_suffix"
remove_prefix = "remove_prefix"
remove_suffix = "remove_suffix"
}
}
}
}
}
)
Expand Down
Loading