Merge pull request #2 from TianyiQ/main

docs: minor fixes

doc_generation/source/index.rst

@@ -3,37 +3,29 @@
    You can adapt this file completely to your liking, but it should at least
    contain the root `toctree` directive.
 
-ProgressGym: Dataset & Experimental Framework for Progress-Aware Alignment
+ProgressGym: Dataset & Experimental Framework for Progress Alignment
 ==========================================================================
 
 Frontier AI systems, including large language models (LLMs), hold increasing influence over the epistemology of human users. Such influence can reinforce prevailing societal values, potentially contributing to the lock-in of misguided moral beliefs and, consequently, the perpetuation of problematic moral practices on a broad scale. We introduce progress alignment as a technical solution to mitigate this imminent risk.
 
-To empower research in progress alignment, we introduce ProgressGym, an experimental 
-framework that codifies real-world progress alignment challenges into concrete benchmarks.
-Specifically, we introduce three core challenges: tracking evolving values (PG-Follow),
-preemptively anticipating moral progress (PG-Predict), and regulating the feedback loop be-
-tween human and AI value shifts (PG-Coevolve).
+To empower research in progress alignment, we introduce ProgressGym, an experimental framework that codifies real-world progress alignment challenges into concrete benchmarks.
+Specifically, we introduce three core challenges: tracking evolving values (PG-Follow), preemptively anticipating moral progress (PG-Predict), and regulating the feedback loop between human and AI value shifts (PG-Coevolve).
 
 Aside from example alignment algorithms (Lifelong RLHF, Lifelong DPO, Extrapolative RLHF, Extrapolative DPO),
-we provide APIs for user-implemented examinees as well as judges, so that you can evaluate your own algorithm in terms of progress-aware alignment, on your custom tasks.
+we provide APIs for user-implemented examinees as well as judges, so that you can evaluate your own algorithm on your custom progress alignment tasks.
 
 Check out the :doc:`usage` section for basic usage.
 
 Contents
 --------
 .. toctree::
    :maxdepth: 2
-   :caption: Contents:
 
    running
    usage
    documentation
-
+   genindex
 
-
-Indices and tables
-==================
-
-* :ref:`genindex`
-* :ref:`modindex`
-* :ref:`search`
+.. * :ref:`genindex`
+.. * :ref:`modindex`
+.. * :ref:`search`

doc_generation/source/running.rst

@@ -39,13 +39,13 @@ Basic Usage
 
 **Building Dataset and Training Models**
 
-*Notice that you don't have to run this part yourself unless you want to re-create our findings in the paper, as the HisText dataset has already been utilized to train the Human Proxy Model sequence.*
+*Note that you do not need to run this part yourself unless you want to replicate our findings, as the HisText dataset has already been utilized to train the HistLlama model sequence, and is available on* `Huggingface <https://huggingface.co/collections/PKU-Alignment/progressgym-666735fcf3e4efa276226eaa>`_.
 
 Running 
 
-.. code-block:: python
+.. code-block:: console
 
-   python ./build_dataset.py 
+   $ python build_dataset.py 
 
 will automatically download the relevant files and then compile them into the desired dataset. Specifically, it will...
   - Download and decompress raw data files from multiple sources, placing them in ``./dataset/raw_downloads``

doc_generation/source/usage.rst

@@ -33,7 +33,7 @@ Our implementations of the three major tasks are in ``./challenges``, for your r
     :members:
     :noindex:
 
-Similarly, use *dummy_examinee* to verify your implementation.
+Similarly, use the dummy examinee to verify your implementation.
 
 .. code-block:: console
 

docs/_sources/index.rst.txt

@@ -3,37 +3,29 @@
    You can adapt this file completely to your liking, but it should at least
    contain the root `toctree` directive.
 
-ProgressGym: Dataset & Experimental Framework for Progress-Aware Alignment
+ProgressGym: Dataset & Experimental Framework for Progress Alignment
 ==========================================================================
 
 Frontier AI systems, including large language models (LLMs), hold increasing influence over the epistemology of human users. Such influence can reinforce prevailing societal values, potentially contributing to the lock-in of misguided moral beliefs and, consequently, the perpetuation of problematic moral practices on a broad scale. We introduce progress alignment as a technical solution to mitigate this imminent risk.
 
-To empower research in progress alignment, we introduce ProgressGym, an experimental 
-framework that codifies real-world progress alignment challenges into concrete benchmarks.
-Specifically, we introduce three core challenges: tracking evolving values (PG-Follow),
-preemptively anticipating moral progress (PG-Predict), and regulating the feedback loop be-
-tween human and AI value shifts (PG-Coevolve).
+To empower research in progress alignment, we introduce ProgressGym, an experimental framework that codifies real-world progress alignment challenges into concrete benchmarks.
+Specifically, we introduce three core challenges: tracking evolving values (PG-Follow), preemptively anticipating moral progress (PG-Predict), and regulating the feedback loop between human and AI value shifts (PG-Coevolve).
 
 Aside from example alignment algorithms (Lifelong RLHF, Lifelong DPO, Extrapolative RLHF, Extrapolative DPO),
-we provide APIs for user-implemented examinees as well as judges, so that you can evaluate your own algorithm in terms of progress-aware alignment, on your custom tasks.
+we provide APIs for user-implemented examinees as well as judges, so that you can evaluate your own algorithm on your custom progress alignment tasks.
 
 Check out the :doc:`usage` section for basic usage.
 
 Contents
 --------
 .. toctree::
    :maxdepth: 2
-   :caption: Contents:
 
    running
    usage
    documentation
-
+   genindex
 
-
-Indices and tables
-==================
-
-* :ref:`genindex`
-* :ref:`modindex`
-* :ref:`search`
+.. * :ref:`genindex`
+.. * :ref:`modindex`
+.. * :ref:`search`

docs/_sources/running.rst.txt

@@ -39,13 +39,13 @@ Basic Usage
 
 **Building Dataset and Training Models**
 
-*Notice that you don't have to run this part yourself unless you want to re-create our findings in the paper, as the HisText dataset has already been utilized to train the Human Proxy Model sequence.*
+*Note that you do not need to run this part yourself unless you want to replicate our findings, as the HisText dataset has already been utilized to train the HistLlama model sequence, and is available on* `Huggingface <https://huggingface.co/collections/PKU-Alignment/progressgym-666735fcf3e4efa276226eaa>`_.
 
 Running 
 
-.. code-block:: python
+.. code-block:: console
 
-   python ./build_dataset.py 
+   $ python build_dataset.py 
 
 will automatically download the relevant files and then compile them into the desired dataset. Specifically, it will...
   - Download and decompress raw data files from multiple sources, placing them in ``./dataset/raw_downloads``

docs/_sources/usage.rst.txt

@@ -33,7 +33,7 @@ Our implementations of the three major tasks are in ``./challenges``, for your r
     :members:
     :noindex:
 
-Similarly, use *dummy_examinee* to verify your implementation.
+Similarly, use the dummy examinee to verify your implementation.
 
 .. code-block:: console
 

docs/genindex.html

@@ -196,8 +196,7 @@
   <input type="hidden" name="area" value="default">
 </form>
 <div id="searchbox"></div><div class="sidebar-scroll"><div class="sidebar-tree">
-  <p class="caption" role="heading"><span class="caption-text">Contents:</span></p>
-<ul>
+  <ul class="current">
 <li class="toctree-l1"><a class="reference internal" href="running.html">Running the benchmark</a></li>
 <li class="toctree-l1"><a class="reference internal" href="usage.html">Implementing Your Own Experiment</a></li>
 <li class="toctree-l1 has-children"><a class="reference internal" href="documentation.html">Documentation</a><input class="toctree-checkbox" id="toctree-checkbox-1" name="toctree-checkbox-1" role="switch" type="checkbox"/><label for="toctree-checkbox-1"><div class="visually-hidden">Toggle navigation of Documentation</div><i class="icon"><svg><use href="#svg-arrow-right"></use></svg></i></label><ul>
@@ -207,6 +206,7 @@
 <li class="toctree-l2"><a class="reference internal" href="Examinee.html">Examinee</a></li>
 </ul>
 </li>
+<li class="toctree-l1 current current-page"><a class="current reference internal" href="#">Index</a></li>
 </ul>
 
 </div>

docs/index.html

@@ -198,8 +198,7 @@
   <input type="hidden" name="area" value="default">
 </form>
 <div id="searchbox"></div><div class="sidebar-scroll"><div class="sidebar-tree">
-  <p class="caption" role="heading"><span class="caption-text">Contents:</span></p>
-<ul>
+  <ul>
 <li class="toctree-l1"><a class="reference internal" href="running.html">Running the benchmark</a></li>
 <li class="toctree-l1"><a class="reference internal" href="usage.html">Implementing Your Own Experiment</a></li>
 <li class="toctree-l1 has-children"><a class="reference internal" href="documentation.html">Documentation</a><input class="toctree-checkbox" id="toctree-checkbox-1" name="toctree-checkbox-1" role="switch" type="checkbox"/><label for="toctree-checkbox-1"><div class="visually-hidden">Toggle navigation of Documentation</div><i class="icon"><svg><use href="#svg-arrow-right"></use></svg></i></label><ul>
@@ -209,6 +208,7 @@
 <li class="toctree-l2"><a class="reference internal" href="Examinee.html">Examinee</a></li>
 </ul>
 </li>
+<li class="toctree-l1"><a class="reference internal" href="genindex.html">Index</a></li>
 </ul>
 
 </div>
@@ -249,21 +249,17 @@
           </label>
         </div>
         <article role="main" id="furo-main-content">
-          <section id="progressgym-dataset-experimental-framework-for-progress-aware-alignment">
-<h1>ProgressGym: Dataset &amp; Experimental Framework for Progress-Aware Alignment<a class="headerlink" href="#progressgym-dataset-experimental-framework-for-progress-aware-alignment" title="Permalink to this heading">¶</a></h1>
+          <section id="progressgym-dataset-experimental-framework-for-progress-alignment">
+<h1>ProgressGym: Dataset &amp; Experimental Framework for Progress Alignment<a class="headerlink" href="#progressgym-dataset-experimental-framework-for-progress-alignment" title="Permalink to this heading">¶</a></h1>
 <p>Frontier AI systems, including large language models (LLMs), hold increasing influence over the epistemology of human users. Such influence can reinforce prevailing societal values, potentially contributing to the lock-in of misguided moral beliefs and, consequently, the perpetuation of problematic moral practices on a broad scale. We introduce progress alignment as a technical solution to mitigate this imminent risk.</p>
-<p>To empower research in progress alignment, we introduce ProgressGym, an experimental
-framework that codifies real-world progress alignment challenges into concrete benchmarks.
-Specifically, we introduce three core challenges: tracking evolving values (PG-Follow),
-preemptively anticipating moral progress (PG-Predict), and regulating the feedback loop be-
-tween human and AI value shifts (PG-Coevolve).</p>
+<p>To empower research in progress alignment, we introduce ProgressGym, an experimental framework that codifies real-world progress alignment challenges into concrete benchmarks.
+Specifically, we introduce three core challenges: tracking evolving values (PG-Follow), preemptively anticipating moral progress (PG-Predict), and regulating the feedback loop between human and AI value shifts (PG-Coevolve).</p>
 <p>Aside from example alignment algorithms (Lifelong RLHF, Lifelong DPO, Extrapolative RLHF, Extrapolative DPO),
-we provide APIs for user-implemented examinees as well as judges, so that you can evaluate your own algorithm in terms of progress-aware alignment, on your custom tasks.</p>
+we provide APIs for user-implemented examinees as well as judges, so that you can evaluate your own algorithm on your custom progress alignment tasks.</p>
 <p>Check out the <a class="reference internal" href="usage.html"><span class="doc">Implementing Your Own Experiment</span></a> section for basic usage.</p>
 <section id="contents">
 <h2>Contents<a class="headerlink" href="#contents" title="Permalink to this heading">¶</a></h2>
 <div class="toctree-wrapper compound">
-<p class="caption" role="heading"><span class="caption-text">Contents:</span></p>
 <ul>
 <li class="toctree-l1"><a class="reference internal" href="running.html">Running the benchmark</a><ul>
 <li class="toctree-l2"><a class="reference internal" href="running.html#requirements">Requirements</a></li>
@@ -276,17 +272,10 @@ <h2>Contents<a class="headerlink" href="#contents" title="Permalink to this head
 <li class="toctree-l2"><a class="reference internal" href="documentation.html#contents">Contents</a></li>
 </ul>
 </li>
+<li class="toctree-l1"><a class="reference internal" href="genindex.html">Index</a></li>
 </ul>
 </div>
 </section>
-</section>
-<section id="indices-and-tables">
-<h1>Indices and tables<a class="headerlink" href="#indices-and-tables" title="Permalink to this heading">¶</a></h1>
-<ul class="simple">
-<li><p><a class="reference internal" href="genindex.html"><span class="std std-ref">Index</span></a></p></li>
-<li><p><a class="reference internal" href="py-modindex.html"><span class="std std-ref">Module Index</span></a></p></li>
-<li><p><a class="reference internal" href="search.html"><span class="std std-ref">Search Page</span></a></p></li>
-</ul>
 </section>
 
         </article>
@@ -334,11 +323,10 @@ <h1>Indices and tables<a class="headerlink" href="#indices-and-tables" title="Pe
         <div class="toc-tree-container">
           <div class="toc-tree">
             <ul>
-<li><a class="reference internal" href="#">ProgressGym: Dataset &amp; Experimental Framework for Progress-Aware Alignment</a><ul>
+<li><a class="reference internal" href="#">ProgressGym: Dataset &amp; Experimental Framework for Progress Alignment</a><ul>
 <li><a class="reference internal" href="#contents">Contents</a></li>
 </ul>
 </li>
-<li><a class="reference internal" href="#indices-and-tables">Indices and tables</a></li>
 </ul>
 
           </div>

docs/running.html

@@ -3,7 +3,7 @@
   <head><meta charset="utf-8"/>
     <meta name="viewport" content="width=device-width,initial-scale=1"/>
     <meta name="color-scheme" content="light dark"><meta name="viewport" content="width=device-width, initial-scale=1" />
-<link rel="index" title="Index" href="genindex.html" /><link rel="search" title="Search" href="search.html" /><link rel="next" title="Implementing Your Own Experiment" href="usage.html" /><link rel="prev" title="ProgressGym: Dataset &amp; Experimental Framework for Progress-Aware Alignment" href="index.html" />
+<link rel="index" title="Index" href="genindex.html" /><link rel="search" title="Search" href="search.html" /><link rel="next" title="Implementing Your Own Experiment" href="usage.html" /><link rel="prev" title="ProgressGym: Dataset &amp; Experimental Framework for Progress Alignment" href="index.html" />
 
     <!-- Generated with Sphinx 7.1.2 and Furo 2024.07.18 -->
         <title>Running the benchmark - ProgressGym 0.1 documentation</title>
@@ -198,8 +198,7 @@
   <input type="hidden" name="area" value="default">
 </form>
 <div id="searchbox"></div><div class="sidebar-scroll"><div class="sidebar-tree">
-  <p class="caption" role="heading"><span class="caption-text">Contents:</span></p>
-<ul class="current">
+  <ul class="current">
 <li class="toctree-l1 current current-page"><a class="current reference internal" href="#">Running the benchmark</a></li>
 <li class="toctree-l1"><a class="reference internal" href="usage.html">Implementing Your Own Experiment</a></li>
 <li class="toctree-l1 has-children"><a class="reference internal" href="documentation.html">Documentation</a><input class="toctree-checkbox" id="toctree-checkbox-1" name="toctree-checkbox-1" role="switch" type="checkbox"/><label for="toctree-checkbox-1"><div class="visually-hidden">Toggle navigation of Documentation</div><i class="icon"><svg><use href="#svg-arrow-right"></use></svg></i></label><ul>
@@ -209,6 +208,7 @@
 <li class="toctree-l2"><a class="reference internal" href="Examinee.html">Examinee</a></li>
 </ul>
 </li>
+<li class="toctree-l1"><a class="reference internal" href="genindex.html">Index</a></li>
 </ul>
 
 </div>
@@ -282,9 +282,9 @@ <h2>Installation<a class="headerlink" href="#installation" title="Permalink to t
 <section id="basic-usage">
 <h2>Basic Usage<a class="headerlink" href="#basic-usage" title="Permalink to this heading">¶</a></h2>
 <p><strong>Building Dataset and Training Models</strong></p>
-<p><em>Notice that you don’t have to run this part yourself unless you want to re-create our findings in the paper, as the HisText dataset has already been utilized to train the Human Proxy Model sequence.</em></p>
+<p><em>Note that you do not need to run this part yourself unless you want to replicate our findings, as the HisText dataset has already been utilized to train the HistLlama model sequence, and is available on</em> <a class="reference external" href="https://huggingface.co/collections/PKU-Alignment/progressgym-666735fcf3e4efa276226eaa">Huggingface</a>.</p>
 <p>Running</p>
-<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">python</span> <span class="o">./</span><span class="n">build_dataset</span><span class="o">.</span><span class="n">py</span>
+<div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>python<span class="w"> </span>build_dataset.py
 </pre></div>
 </div>
 <dl class="simple">

docs/search.html

@@ -198,8 +198,7 @@
   <input type="hidden" name="area" value="default">
 </form>
 <div id="searchbox"></div><div class="sidebar-scroll"><div class="sidebar-tree">
-  <p class="caption" role="heading"><span class="caption-text">Contents:</span></p>
-<ul>
+  <ul>
 <li class="toctree-l1"><a class="reference internal" href="running.html">Running the benchmark</a></li>
 <li class="toctree-l1"><a class="reference internal" href="usage.html">Implementing Your Own Experiment</a></li>
 <li class="toctree-l1 has-children"><a class="reference internal" href="documentation.html">Documentation</a><input class="toctree-checkbox" id="toctree-checkbox-1" name="toctree-checkbox-1" role="switch" type="checkbox"/><label for="toctree-checkbox-1"><div class="visually-hidden">Toggle navigation of Documentation</div><i class="icon"><svg><use href="#svg-arrow-right"></use></svg></i></label><ul>
@@ -209,6 +208,7 @@
 <li class="toctree-l2"><a class="reference internal" href="Examinee.html">Examinee</a></li>
 </ul>
 </li>
+<li class="toctree-l1"><a class="reference internal" href="genindex.html">Index</a></li>
 </ul>
 
 </div>