configure-autoawq.md

Configure ModelSmith with AutoAWQ

This guide provides instructions to configure ModelSmith to work with AutoAWQ, facilitating the process of model quantization using remote computational resources.

References

• Original Paper: AutoAWQ
• Main Repository: AutoAWQ

Prerequisites

Before proceeding, ensure you have configured the project locally or on a VM.

1. Activate the current Conda Environment:

   conda activate modelsmith

2. Install Required Libraries:

   Navigate to the autoawq directory and install AutoAWQ:

   cd /path/to/machine_learning_core/autoawq
   pip install autoawq

3. Configure AutoAWQ:

   In this step, you need to obtain an access token from Hugging Face and add it to your .env configuration file.

   1. Obtain the Access Token:

      • If you don't have a Hugging Face account, create one here.
      • Request access to Mistral-7B-Instruct-v0.2 on Hugging Face.
      • After gaining access, navigate to your Hugging Face account:
        • Go to Profile/Settings.
        • Under the "Access Tokens" section, generate a new token with read permissions.

   2. Add the Access Token to the .env File:

      Open your .env file and add the following line, replacing insert_token_here with your actual Hugging Face access token:

      HUGGING_FACE_ACCESS_TOKEN=insert_token_here

For further details on configuring AutoAWQ, refer to the AutoAWQ documentation.