Skip to content

Commit

Permalink
Merge pull request #134 from meghdadFar/feature/chat-in-streamlit
Browse files Browse the repository at this point in the history
Chat functionality and chat UI
  • Loading branch information
meghdadFar authored Apr 4, 2024
2 parents bfa7125 + b13a13f commit 7622109
Show file tree
Hide file tree
Showing 21 changed files with 766 additions and 261 deletions.
5 changes: 5 additions & 0 deletions CHANGES.rst
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
Version 1.3.0
-------------
- Chat feature for text analysis.


Version 1.2.0
-------------
- Support for all Penn POS tags
Expand Down
12 changes: 10 additions & 2 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -89,15 +89,23 @@ of interest where we endeavor to find them in order to shed light into our data.
Wordview offers several anomaly and outlier detection functions.



Cluster Analysis
****************
Clustering can be used to identify different groups of documents with similar information, in an unsupervised fashion.
Despite it's ability to provide valuable insights into your data, you do not need labeled data for clustering.



Chat with Wordview
******************
Worldview integrates GPT-3.5-Turbo to enable the users to interact with their data and get insights from it via Natural Language.

.. image:: sphinx-docs/figs/chat.png
:width: 100%
:align: center

Utilities
#########
*********
Wordview offers a number of utility functions that you can use for common pre and post processing tasks in NLP.

Contributing
Expand Down
Binary file modified docs/.doctrees/api.doctree
Binary file not shown.
Binary file added docs/.doctrees/chat.doctree
Binary file not shown.
Binary file modified docs/.doctrees/contributing.doctree
Binary file not shown.
Binary file added docs/_images/chat.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
32 changes: 32 additions & 0 deletions docs/_sources/chat.rst.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
Chat with Wordview
##################

Worldview integrates GPT-3.5-Turbo to enable the users to interact with their data and get insights from it via Natural Language.
See examples of chatting with different components of Wordview below.


Chat with TextStatsPlots
~~~~~~~~~~~~~~~~~~~~~~~~

After allowing Wordview to process and analyze your corpus, you can
call the `chat` method to interact with the data and get insights from it via Natural Language.

.. code:: python
import json
import pandas as pd
from wordview.text_analysis import TextStatsPlots
imdb_df = pd.read_csv("data/IMDB_Dataset_sample_5k.csv")
with open("wordview/chat/secrets/openai_api_key.json", "r") as f:
credentials = json.load(f)
tsp = TextStatsPlots(df=imdb_df, text_column="review")
tsp.chat(api_key=credentials.get("openai_api_key"))
The chat UI is available under http://127.0.0.1:5000/

|chat|

.. |chat| image:: ../figs/chat.png
32 changes: 31 additions & 1 deletion docs/api.html
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,8 @@
<li class="toctree-l2"><a class="reference internal" href="#text-analysis">Text Analysis</a><ul>
<li class="toctree-l3"><a class="reference internal" href="#wordview.text_analysis.TextStatsPlots"><code class="docutils literal notranslate"><span class="pre">TextStatsPlots</span></code></a><ul>
<li class="toctree-l4"><a class="reference internal" href="#wordview.text_analysis.TextStatsPlots.__init__"><code class="docutils literal notranslate"><span class="pre">TextStatsPlots.__init__()</span></code></a></li>
<li class="toctree-l4"><a class="reference internal" href="#wordview.text_analysis.TextStatsPlots.chat"><code class="docutils literal notranslate"><span class="pre">TextStatsPlots.chat()</span></code></a></li>
<li class="toctree-l4"><a class="reference internal" href="#wordview.text_analysis.TextStatsPlots.return_stats"><code class="docutils literal notranslate"><span class="pre">TextStatsPlots.return_stats()</span></code></a></li>
<li class="toctree-l4"><a class="reference internal" href="#wordview.text_analysis.TextStatsPlots.show_bar_plots"><code class="docutils literal notranslate"><span class="pre">TextStatsPlots.show_bar_plots()</span></code></a></li>
<li class="toctree-l4"><a class="reference internal" href="#wordview.text_analysis.TextStatsPlots.show_distplot"><code class="docutils literal notranslate"><span class="pre">TextStatsPlots.show_distplot()</span></code></a></li>
<li class="toctree-l4"><a class="reference internal" href="#wordview.text_analysis.TextStatsPlots.show_insights"><code class="docutils literal notranslate"><span class="pre">TextStatsPlots.show_insights()</span></code></a></li>
Expand Down Expand Up @@ -191,6 +193,35 @@ <h2>Text Analysis<a class="headerlink" href="#text-analysis" title="Permalink to
</dl>
</dd></dl>

<dl class="py method">
<dt class="sig sig-object py" id="wordview.text_analysis.TextStatsPlots.chat">
<span class="sig-name descname"><span class="pre">chat</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">api_key</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">str</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">''</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#wordview.text_analysis.TextStatsPlots.chat" title="Permalink to this definition"></a></dt>
<dd><p>Chat with OpenAI’s latest model about the results of Wordview’s text analysis.
Access the chat UI in your localhost under <a class="reference external" href="http://127.0.0.1:5000/">http://127.0.0.1:5000/</a></p>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><p><strong>api_key</strong> – OpenAI API key.</p>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>None</p>
</dd>
</dl>
</dd></dl>

<dl class="py method">
<dt class="sig sig-object py" id="wordview.text_analysis.TextStatsPlots.return_stats">
<span class="sig-name descname"><span class="pre">return_stats</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">&#x2192;</span> <span class="sig-return-typehint"><span class="pre">dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><span class="pre">Any</span><span class="p"><span class="pre">]</span></span></span></span><a class="headerlink" href="#wordview.text_analysis.TextStatsPlots.return_stats" title="Permalink to this definition"></a></dt>
<dd><p>Returns dataset statistics, including:
Language/s
Number of unique words
Number of all words
Number of documents
Median document length
Number of nouns
Number of adjectives
Number of verbs.</p>
</dd></dl>

<dl class="py method">
<dt class="sig sig-object py" id="wordview.text_analysis.TextStatsPlots.show_bar_plots">
<span class="sig-name descname"><span class="pre">show_bar_plots</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">pos</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">str</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">layout_settings</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><span class="pre">Any</span><span class="p"><span class="pre">]</span></span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">{}</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">plot_settings</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><span class="pre">str</span><span class="p"><span class="pre">]</span></span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">{}</span></span></em><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">&#x2192;</span> <span class="sig-return-typehint"><span class="pre">None</span></span></span><a class="headerlink" href="#wordview.text_analysis.TextStatsPlots.show_bar_plots" title="Permalink to this definition"></a></dt>
Expand Down Expand Up @@ -391,7 +422,6 @@ <h2>Extraction &amp; Analysis of Multiword Expressions<a class="headerlink" href
NP: {&lt;DT&gt;?&lt;JJ&gt;*&lt;NN&gt;} # Noun phrase
You can use multiple and/or nested patterns, separated by a newline character e.g.:
custom_pattern = ‘’’
NP: {&lt;DT&gt;?&lt;JJ&gt;*&lt;NN&gt;} # Noun phrase
VP: {&lt;MD&gt;?&lt;VB.*&gt;&lt;NP|PP|CLAUSE&gt;+$} # Verb phrase
PROPN: {&lt;NNP&gt;+} # Proper noun
ADJP: {&lt;RB|RBR|RBS&gt;*&lt;JJ&gt;} # Adjective phrase
Expand Down
152 changes: 152 additions & 0 deletions docs/chat.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,152 @@
<!DOCTYPE html>
<html class="writer-html5" lang="en" >
<head>
<meta charset="utf-8" /><meta name="generator" content="Docutils 0.18.1: http://docutils.sourceforge.net/" />

<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Chat with Wordview &mdash; wordview &#34;1.1.2&#34; documentation</title>
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="_static/css/theme.css" type="text/css" />
<link rel="stylesheet" href="_static/custom.css" type="text/css" />
<link rel="stylesheet" href="_static/fonts.css" type="text/css" />
<!--[if lt IE 9]>
<script src="_static/js/html5shiv.min.js"></script>
<![endif]-->

<script src="_static/jquery.js"></script>
<script src="_static/_sphinx_javascript_frameworks_compat.js"></script>
<script data-url_root="./" id="documentation_options" src="_static/documentation_options.js"></script>
<script src="_static/doctools.js"></script>
<script src="_static/sphinx_highlight.js"></script>
<script src="_static/js/theme.js"></script>
<link rel="index" title="Index" href="genindex.html" />
<link rel="search" title="Search" href="search.html" />
</head>

<body class="wy-body-for-nav">
<div class="wy-grid-for-nav">
<nav data-toggle="wy-nav-shift" class="wy-nav-side">
<div class="wy-side-scroll">
<div class="wy-side-nav-search" >



<a href="index.html" class="icon icon-home">
wordview
</a>
<div role="search">
<form id="rtd-search-form" class="wy-form" action="search.html" method="get">
<input type="text" name="q" placeholder="Search docs" aria-label="Search docs" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
</div><div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="Navigation menu">
<p class="caption" role="heading"><span class="caption-text">Quick Start</span></p>
<ul>
<li class="toctree-l1"><a class="reference internal" href="start.html">Installation</a></li>
<li class="toctree-l1"><a class="reference internal" href="start.html#what-is-wordview">What is Wordview</a></li>
<li class="toctree-l1"><a class="reference internal" href="start.html#tutorials">Tutorials</a></li>
</ul>
<p class="caption" role="heading"><span class="caption-text">Exploratory Data Analysis (EDA)</span></p>
<ul>
<li class="toctree-l1"><a class="reference internal" href="textstats.html">Text Analysis</a></li>
<li class="toctree-l1"><a class="reference internal" href="labels.html">Label Analysis</a></li>
<li class="toctree-l1"><a class="reference internal" href="mwes.html">Analysis &amp; Extraction of Multiword Expressions (MWEs)</a></li>
<li class="toctree-l1"><a class="reference internal" href="bias.html">Bias Analysis</a></li>
<li class="toctree-l1"><a class="reference internal" href="anomalies.html">Analysis of Anomalies &amp; Outliers</a></li>
<li class="toctree-l1"><a class="reference internal" href="clustering.html">Cluster Analysis</a></li>
</ul>
<p class="caption" role="heading"><span class="caption-text">Utilities</span></p>
<ul>
<li class="toctree-l1"><a class="reference internal" href="utilities.html">Utilities</a></li>
</ul>
<p class="caption" role="heading"><span class="caption-text">Contributing</span></p>
<ul>
<li class="toctree-l1"><a class="reference internal" href="contributing.html">Contributing to wordview</a></li>
<li class="toctree-l1"><a class="reference internal" href="codeofconduct.html">Contributor Code of Conduct</a></li>
</ul>
<p class="caption" role="heading"><span class="caption-text">API Reference</span></p>
<ul>
<li class="toctree-l1"><a class="reference internal" href="api.html">API</a></li>
</ul>

</div>
</div>
</nav>

<section data-toggle="wy-nav-shift" class="wy-nav-content-wrap"><nav class="wy-nav-top" aria-label="Mobile navigation menu" >
<i data-toggle="wy-nav-top" class="fa fa-bars"></i>
<a href="index.html">wordview</a>
</nav>

<div class="wy-nav-content">
<div class="rst-content">
<div role="navigation" aria-label="Page navigation">
<ul class="wy-breadcrumbs">
<li><a href="index.html" class="icon icon-home" aria-label="Home"></a></li>
<li class="breadcrumb-item active">Chat with Wordview</li>
<li class="wy-breadcrumbs-aside">
<a href="_sources/chat.rst.txt" rel="nofollow"> View page source</a>
</li>
</ul>
<hr/>
</div>
<div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
<div itemprop="articleBody">

<section id="chat-with-wordview">
<h1>Chat with Wordview<a class="headerlink" href="#chat-with-wordview" title="Permalink to this heading"></a></h1>
<p>Worldview integrates GPT-3.5-Turbo to enable the users to interact with their data and get insights from it via Natural Language.
See examples of chatting with different components of Wordview below.</p>
<section id="chat-with-textstatsplots">
<h2>Chat with TextStatsPlots<a class="headerlink" href="#chat-with-textstatsplots" title="Permalink to this heading"></a></h2>
<p>After allowing Wordview to process and analyze your corpus, you can
call the <cite>chat</cite> method to interact with the data and get insights from it via Natural Language.</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">json</span>

<span class="kn">import</span> <span class="nn">pandas</span> <span class="k">as</span> <span class="nn">pd</span>

<span class="kn">from</span> <span class="nn">wordview.text_analysis</span> <span class="kn">import</span> <span class="n">TextStatsPlots</span>
<span class="n">imdb_df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_csv</span><span class="p">(</span><span class="s2">&quot;data/IMDB_Dataset_sample_5k.csv&quot;</span><span class="p">)</span>
<span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="s2">&quot;wordview/chat/secrets/openai_api_key.json&quot;</span><span class="p">,</span> <span class="s2">&quot;r&quot;</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
<span class="n">credentials</span> <span class="o">=</span> <span class="n">json</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="n">f</span><span class="p">)</span>

<span class="n">tsp</span> <span class="o">=</span> <span class="n">TextStatsPlots</span><span class="p">(</span><span class="n">df</span><span class="o">=</span><span class="n">imdb_df</span><span class="p">,</span> <span class="n">text_column</span><span class="o">=</span><span class="s2">&quot;review&quot;</span><span class="p">)</span>
<span class="n">tsp</span><span class="o">.</span><span class="n">chat</span><span class="p">(</span><span class="n">api_key</span><span class="o">=</span><span class="n">credentials</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;openai_api_key&quot;</span><span class="p">))</span>
</pre></div>
</div>
<p>The chat UI is available under <a class="reference external" href="http://127.0.0.1:5000/">http://127.0.0.1:5000/</a></p>
<p><img alt="chat" src="_images/chat.png" /></p>
</section>
</section>


</div>
</div>
<footer>

<hr/>

<div role="contentinfo">
<p>&#169; Copyright 2023, Meghdad Farahmand.</p>
</div>

Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.


</footer>
</div>
</div>
</section>
</div>
<script>
jQuery(function () {
SphinxRtdTheme.Navigation.enable(true);
});
</script>

</body>
</html>
11 changes: 11 additions & 0 deletions docs/contributing.html
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,7 @@
<li class="toctree-l2"><a class="reference internal" href="#setup-your-dev-environment">Setup your Dev Environment</a></li>
<li class="toctree-l2"><a class="reference internal" href="#testing">Testing</a></li>
<li class="toctree-l2"><a class="reference internal" href="#code-quality">Code Quality</a></li>
<li class="toctree-l2"><a class="reference internal" href="#building-the-documentation">Building the Documentation</a></li>
<li class="toctree-l2"><a class="reference internal" href="#pull-request-pr">Pull Request (PR)</a></li>
</ul>
</li>
Expand Down Expand Up @@ -210,6 +211,16 @@ <h2>Code Quality<a class="headerlink" href="#code-quality" title="Permalink to t
skip pre-commit checks by running your <code class="docutils literal notranslate"><span class="pre">git</span> <span class="pre">commit</span></code> with the <code class="docutils literal notranslate"><span class="pre">--no-verify</span></code> flag (e.g. <code class="docutils literal notranslate"><span class="pre">git</span> <span class="pre">commit</span> <span class="pre">-m</span> <span class="pre">'dirty</span> <span class="pre">fix'</span> <span class="pre">--no-verify</span></code>), however,
this is discouraged unless you really have to.</p>
</section>
<section id="building-the-documentation">
<h2>Building the Documentation<a class="headerlink" href="#building-the-documentation" title="Permalink to this heading"></a></h2>
<p>We use <a class="reference external" href="https://www.sphinx-doc.org/">Sphinx</a> to build the documentation.</p>
<p>When you create new functions, write new docstring, or change the existing one, or in general change the documentation, you need to build the documentation again. To do so, run the following command:</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>sphinx-build<span class="w"> </span>-b<span class="w"> </span>html<span class="w"> </span>sphinx-docs/source<span class="w"> </span>docs/
</pre></div>
</div>
<p>Also, ensure that the file <cite>.nojekyll</cite> always exists in the docs directory otherwise, the docs will not render correctly on GitHub Pages.</p>
<p>This will build the documentation in the <a class="reference external" href="docs/">docs</a> directory. You can now commit and push your changes to remote.</p>
</section>
<section id="pull-request-pr">
<h2>Pull Request (PR)<a class="headerlink" href="#pull-request-pr" title="Permalink to this heading"></a></h2>
<p>Once your work is complete, you can make a pull request. Remember to
Expand Down
Loading

0 comments on commit 7622109

Please sign in to comment.