first commit

instruction/Plugin-Roles.md

@@ -0,0 +1,94 @@
+## DetectSceneLabels
+
+- Attach policy: `AmazonBedrockFullAccess`
+- Add the following policy in `DetectSceneLabelsRoleDefaultPolicy`:
+
+```
+{
+    "Version": "2012-10-17",
+    "Statement": [
+        ...,
+        {
+            "Action": [
+                "dynamodb:DeleteItem",
+                "dynamodb:PutItem",
+                "dynamodb:GetItem",
+                "dynamodb:Scan",
+                "dynamodb:Query",
+                "dynamodb:UpdateItem",
+                "dynamodb:UpdateTable",
+                "dynamodb:GetRecords"
+            ],
+            "Resource": "*",
+            "Effect": "Allow"
+        }
+    ]
+}
+```
+
+## DetectCelebrities
+
+- Attach policy: `AmazonBedrockFullAccess`
+- Add the following policy in `DetectCelebritiesRoleDefaultPolicy`:
+
+```
+{
+    "Version": "2012-10-17",
+    "Statement": [
+        ...
+        {
+            "Action": "rekognition:*",
+            "Resource": "*",
+            "Effect": "Allow"
+        },
+        {
+            "Sid": "VisualEditor0",
+            "Effect": "Allow",
+            "Action": [
+                "dynamodb:PutItem",
+                "dynamodb:DeleteItem",
+                "dynamodb:GetItem",
+                "dynamodb:Scan",
+                "dynamodb:Query",
+                "dynamodb:UpdateItem",
+                "rekognition:*",
+                "dynamodb:UpdateTable",
+                "dynamodb:GetRecords"
+            ],
+            "Resource": "*"
+        }
+    ]
+}
+```
+
+## SegmentNewsRole
+- Attach policy: `AmazonBedrockFullAccess`
+- Add the following policy in `SegmentNewsRoleDefaultPolicy`:
+
+```
+{
+    "Version": "2012-10-17",
+    "Statement": [
+        {
+            "Effect": "Allow",
+            "Action": "aoss:APIAccessAll",
+            "Resource": "*"
+        },
+        {
+            "Action": [
+                "dynamodb:DeleteItem",
+                "dynamodb:PutItem",
+                "dynamodb:GetItem",
+                "dynamodb:Scan",
+                "dynamodb:Query",
+                "dynamodb:UpdateItem",
+                "dynamodb:UpdateTable",
+                "dynamodb:GetRecords"
+            ],
+            "Resource": "*",
+            "Effect": "Allow"
+        },
+        ...
+    ]
+}
+```

instruction/README.md

@@ -0,0 +1,104 @@
+# MRE - WizeLine instruction
+
+## Prerequisites
+
+* python == 3.11
+* aws-cli
+* aws-cdk >= 2.24.1
+* docker
+* node >= 20.10.0
+* npm >= 10.2.3
+* git
+
+## How to install MRE
+
+## Step 1: Install MRE Core
+
+```bash
+REGION=[specify the AWS region. For example, us-east-1]
+VERSION=2.9.0
+git clone https://git-codecommit.us-east-1.amazonaws.com/v1/repos/aws-media-replay-engine-gen-ai
+git checkout feat/wl-main
+cd aws-media-replay-engine-gen-ai
+cd deployment
+./build-and-deploy.sh --enable-ssm-high-throughput --enable-generative-ai --version $VERSION --region $REGION [--profile <aws-profile>]
+```
+
+## Step 2: Install MRE's plugins
+
+1. Install necessasry lambda layers at `labmda-layers`:
+
+```bash
+cd lambda-layers
+./deploy.sh [aws-profile] [aws-region]
+```
+
+1. Replace plugin list in `aws-media-replay-engine-gen-ai/samples/source/mre-plugin-samples/Plugins` with the plugins in `sample-plugins` folder.
+2. Replace the following data into `aws-media-replay-engine-gen-ai/samples/source/mre-plugin-samples/cdk/cdk.context.json`
+
+```json
+{
+    "Plugins": [
+        "DetectActivePresenter",
+        "DetectAudioPeaks",
+        "DetectCameraScene",
+        "DetectCelebrities",
+        "DetectPassThrough100",
+        "DetectSceneLabels",
+        "DetectSentiment",
+        "DetectSpeech",
+        "DetectTennisScoreBoxData",
+        "LabelBasic",
+        "LabelNews",
+        "LabelPassThrough",
+        "LabelTennisScore",
+        "OptimizePassThrough",
+        "SegmentByKeyMoment",
+        "SegmentBySceneChange",
+        "SegmentNews",
+        "SegmentPassThrough100",
+        "DetectKeyContent",
+        "SegmentNewsNotification"
+    ]
+}
+```
+
+3. Run command:
+
+```bash
+cd aws-media-replay-engine-gen-ai/samples/deployment
+./build-and-deploy.sh --app plugin-samples --region $REGION [--profile <aws-profile>]
+```
+
+## Step 4: Plugin configuration: (WIP)
+
+**Many (or all) steps in this section can be automated in script file, but at the point when this document is written, we don't have time and resource to update the script**
+
+1. [Add plugin roles](Plugin-Roles.md)
+2. Create the following S3 buckets:
+
+  - `mre-transcribe-files-wizeline-[poc name]`
+  - `mre-voice-samples-training-wizeline-[poc name]`
+  - `mre-wizeline-video-samples-[poc name]`: Enable Bucket Versioning
+
+3. Add `DetectSpeech` new version with bucket name `mre-transcribe-files-wizeline-[poc name]` and `mre-voice-samples-training-wizeline-[poc name]`:
+
+    ![DetectSpeechConfig](assets/DetectSpeechConfig.png)
+
+4. Create `temp-chunk-vars` table in dynamodb
+5. Request model access in Bedrock:
+   - Titan Embeddings G1 - Text
+   - Titan Text G1 - Express
+   - Titan Text Embeddings V2
+   - Claude 3 Sonnet
+   - Claude 3 Haiku
+   - Claude
+   - Claude Instant
+   - Embed English
+
+## Step 5: Install `live-news-segmenter` app
+
+```bash
+cd aws-media-replay-engine-gen-ai/samples/deployment
+./build-and-deploy.sh --app [live-news-segmenter || live-news-segmenter-ui || live-news-segmenter-api] --region $REGION [--profile <aws-profile>]
+```

lambda-layers/README.md

@@ -0,0 +1,16 @@
+# Lambda Layers
+
+Contains packages of lambda layers needed to install MRE's sample plugins
+
+## Tool Versions
+
+Python3 >= 3.11
+
+## Lambda layer deployment
+
+```bash
+cd lambda-layers
+./deploy.sh [aws-profile] [aws-region]
+```
+
+*(**Note**: if you have permission issue running `./deploy.sh`, you can run `chmod +x deploy.sh` to make it executable.)*

lambda-layers/deploy.sh

@@ -0,0 +1,35 @@
+#!/bin/bash
+
+# $1 refers to the aws-profile argument passed to the script
+AWS_PROFILE=$1
+
+# $2 refers to the AWS region argument passed to the script
+if [ -z "$2" ]; then
+    AWS_DEFAULT_REGION="us-east-1"
+else
+    AWS_DEFAULT_REGION="$2"
+fi
+
+export AWS_PROFILE
+export AWS_DEFAULT_REGION
+
+if [ -d "venv" ]; then
+    echo "Removing existing virtual environment..."
+    rm -rf venv
+fi
+
+echo "Creating new virtual environment..."
+python -m venv venv
+
+echo "Activating virtual environment..."
+source venv/bin/activate
+
+echo "Installing dependencies..."
+pip install -r requirements.txt
+
+echo "Running script..."
+
+# Setup a trap to deactivate the virtual environment upon script exit
+trap "echo 'Deactivating virtual environment...'; deactivate" EXIT
+
+python lambda-layers.py

lambda-layers/lambda-layers.py

@@ -0,0 +1,65 @@
+import boto3
+import json
+import sys
+
+client = boto3.client('lambda')
+
+def create_lambda_layers_from_file(file_path):
+    with open(file_path, 'r') as file:
+        layer_info = json.load(file)
+
+        for layer in layer_info:
+            print(layer)
+            layer_name = layer['name']
+            compatible_runtimes = layer['compatible_runtimes']
+            compatible_architectures = layer['compatible_architectures']
+
+            layer_zip_file = f'./layers/{layer_name}.zip'
+
+            create_lambda_layer(layer_name, layer_zip_file, compatible_runtimes, compatible_architectures)
+
+        return 1
+
+def create_lambda_layer(layer_name, layer_zip_file, compatible_runtimes, compatible_architectures):
+    with open(layer_zip_file, 'rb') as file:
+        layer_zip = file.read()
+
+        if len(compatible_runtimes) > 0 and len(compatible_architectures) > 0:   
+            response = client.publish_layer_version(
+                LayerName=layer_name,
+                Content={
+                    'ZipFile': layer_zip
+                },
+                CompatibleRuntimes=[compatible_runtimes],
+                CompatibleArchitectures=[compatible_architectures]
+            )
+        elif len(compatible_runtimes) > 0 and len(compatible_architectures) == 0:
+            response = client.publish_layer_version(
+                LayerName=layer_name,
+                Content={
+                    'ZipFile': layer_zip
+                },
+                CompatibleRuntimes=[compatible_runtimes]
+            )
+        elif len(compatible_runtimes) == 0 and len(compatible_architectures) > 0:
+            response = client.publish_layer_version(
+                LayerName=layer_name,
+                Content={
+                    'ZipFile': layer_zip
+                },
+                CompatibleArchitectures=[compatible_architectures]
+            )
+
+        layer_arn = response['LayerVersionArn']
+        print(f"Lambda layer '{layer_name}' created with ARN: {layer_arn}")
+
+
+
+if __name__ == '__main__':
+    layer_info_file = './layer-info.json'
+    result = create_lambda_layers_from_file(layer_info_file)
+
+    sys.exit(0) if result else sys.exit(1)
+
+
+

lambda-layers/layer-info.json

@@ -0,0 +1,32 @@
+[
+    {
+        "name": "boto3",
+        "compatible_runtimes": "python3.11",
+        "compatible_architectures": "x86_64"
+    },
+    {
+        "name": "ffmpeg",
+        "compatible_runtimes": "python3.11",
+        "compatible_architectures": "x86_64"
+    },
+    {
+        "name": "json-repair",
+        "compatible_runtimes": "python3.11",
+        "compatible_architectures": "x86_64"
+    },
+    {
+        "name": "opensearch-py",
+        "compatible_runtimes": "python3.11",
+        "compatible_architectures": "x86_64"
+    },
+    {
+        "name": "Pillow",
+        "compatible_runtimes": "python3.11",
+        "compatible_architectures": ""
+    },
+    {
+        "name": "pyAV",
+        "compatible_runtimes": "python3.11",
+        "compatible_architectures": ""
+    }
+]

lambda-layers/requirements.txt

@@ -0,0 +1 @@
+boto3<2.0.0