App for quickly and easily getting indicators into the platform and adding metadata around them.
NOTE: In the past, this app has been focused on importing unstructured content. Moving forward, this app will be updated to handle both structured (e.g. csv and json) and unstructured content.
To install this app in ThreatConnect, use the following commands:
# clone this repo
git clone https://gitlab.com/fhightower-tc/tc-indicator-importer.git
cd tc-indicator-importer
# package the app
make pack
After running these commands, there will be a zip file named TCS_-_Indicator_Importer.zip in the top directory of the app which you can install in ThreatConnect.
Once the app is installed in your TC instance (see administration guide and Installation section), you may have to add the app to your user account's "spaces":
- When logged into ThreatConnect, click the "Spaces" button at the top menu (not Add Menu Space)
- Click "ADD NEW SPACE" button
- Give the space a name and click "SAVE"
- Click "Add App" button
- Select "TCS - Indicator Importer", give it another name (it can be the same name you previously used), and click "SAVE"
Here is a demo setting up the app for the first time:
The indicator importer spaces app pulls attributes from the datastore. Thus, you will need to write a list of attributes like the one in example_datastore_attributes.json to the datastore. This attributes json needs to be added in the following datastore:
/organization/app-data/attributes
{"text": "[...<ATTRIBUTES HERE>...]"}
There is a playbook to do this here.
To be able to import content from a PDF online, you will need to install a playbook with an HTTP link trigger app that makes a request to the given site, reads the contents as a PDF, and returns the text. This playbook is available here. Once you have installed the playbook, you also need to add the link to the playbook's HTTP link trigger into this app's configuration in the field labeled "Link to PDF Reader Playbook".
To be able to import content from a URL, you will need to install a playbook with the HTML Text Parser playbook app. This playbook is available here. Once you have installed the playbook, you also need to add the link to the playbook's HTTP link trigger into this app's configuration in the field labeled "Link to URL Reader Playbook".
To be able to import content from an image online using OCR, you will need to install a playbook with an HTTP link trigger app that makes a request to the given site, runs OCR on the image, and returns the text. Once you have installed such a playbook, you also need to add the link to the playbook's HTTP link trigger into this app's configuration in the field labeled "Link to OCR Playbook".
Every time this app is run, it posts content to a custom metric endpoint. If you're interested in learning more about this, raise an issue and I can explain it further.
This app provides a functionality I'm calling "Indicator Profiles". It allows you to form a template made up of attributes and tags that can be applied to indicators in bulk. For example, I may want to have a profile for callback URLs that applies the "C2" tag and an "IP and Host Usage" attribute with a value of "C2". If you have questions, raise and issue and I can add more documentation about how to make these.
- Add handling for CSV (see our progress here)
- Add ability to edit content (see #18)
- Improve deletion functions (see #19)
- Add ability to parse other structured data types
- Build out documentation
(we are using semantic versioning)
- Renaming the app to reflect the fact that we are handling both structured and unstructured content
- The first version since we have started keeping a changelog
This package was created with Cookiecutter and Floyd Hightower's Spaces App Template.