Video Transcriber and Translator

A Python script that automates video subtitle creation & translation, supporting both local files and online video URLs.

🚀 Features

• Download videos from various platforms (YouTube, Twitter, Facebook, Instagram, etc.)
• Transcribe audio using ChatGPT's Whisper model (default) or local Whisper model
• Translate transcriptions to a target language using OpenAI's GPT models
• Add translated subtitles to videos
• Support for multiple languages and resolutions (including YouTube Shorts)

🎬 Demo

Here are some examples of the script in action:

1. English to Korean Translation

2. English to Spanish Translation

3. German to English Translation

4. Spanish to English Translation

Note: Results may vary with accents or background music, especially when using the local model. Video and caption synchronization might be affected when the audio isn't clear. The above clips were processed using OpenAI for both transcription & translation (default script behavior).

🛠️ Installation

1. Clone the repository:

   git clone https://github.com/tikene/video-caption-and-translate.git
   cd video-caption-and-translate

2. Install required dependencies:

   pip install openai faster_whisper yt-dlp ffmpeg-python

3. Install FFmpeg:

   • Download from ffmpeg.org
   • Add to your system PATH

4. Set up your OpenAI API key:

   export OPENAI_API_KEY='your-api-key-here'

🖥️ Usage

Run the script using the following command:

python translate.py video_input target_language [options]

Arguments:

• video_input: URL or path to the input video file
• target_language: Target language for translation (e.g., Spanish, English, French)

Options:

• --output_dir: Directory to save output files (default: "output")
• --models_path: Path to store Whisper models (default: "Models")
• --openai_api_key: OpenAI API key (if not set as an environment variable)
• --font: Font to use for subtitles (default: "NanumGothic")
• --use_local_whisper: Use local Whisper model for transcription instead of ChatGPT's Whisper

📋 Examples

1. Translate YouTube video subtitles to Spanish (using default ChatGPT Whisper):

   python translate.py https://www.youtube.com/watch?v=VIDEO_ID Spanish

2. Translate local video file subtitles to French (using default ChatGPT Whisper):

   python translate.py /path/to/your/video.mp4 French

3. Use a specific output directory and font:

   python translate.py input_video.mp4 German --output_dir my_output --font Arial

4. Use a local model for transcription:

   python translate.py input_video.mp4 Korean --use_local_whisper

📂 Output

The script generates the following files in the output directory:

1. Downloaded video (if URL was provided)
2. Translated SRT subtitle file
3. Video with embedded translated subtitles

⚠️ Important Notes

• Ensure you have a valid OpenAI API key with sufficient credits.
• The script uses the GPT-4o model for translation, which may incur costs on your OpenAI account (approximately 1-3 cents for a two-minute video).
• Large videos may not work and/or significantly increase costs. Testing has been limited to short videos (up to 5 minutes in length).

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.