Upload 8 files

.gitignore

@@ -0,0 +1,13 @@
+__pycache__/
+.ipynb_checkpoints/
+processed
+outputs
+outputs_v2
+checkpoints
+checkpoints_v2
+trash
+examples*
+.env
+build
+*.egg-info/
+*.zip

LICENSE

@@ -0,0 +1,7 @@
+Copyright 2024 MyShell.ai
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

README.md

@@ -1,13 +1,70 @@
----
-title: Openvoice Api
-emoji: 🦀
-colorFrom: red
-colorTo: green
-sdk: gradio
-sdk_version: 5.34.2
-app_file: app.py
-pinned: false
-short_description: 'Short Voice over agents '
----
-
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+<div align="center">
+  <div>&nbsp;</div>
+  <img src="resources/openvoicelogo.jpg" width="400"/> 
+
+[Paper](https://arxiv.org/abs/2312.01479) |
+[Website](https://research.myshell.ai/open-voice) <br> <br>
+<a href="https://trendshift.io/repositories/6161" target="_blank"><img src="https://trendshift.io/api/badge/repositories/6161" alt="myshell-ai%2FOpenVoice | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
+</div>
+
+## Introduction
+
+### OpenVoice V1
+
+As we detailed in our [paper](https://arxiv.org/abs/2312.01479) and [website](https://research.myshell.ai/open-voice), the advantages of OpenVoice are three-fold:
+
+**1. Accurate Tone Color Cloning.**
+OpenVoice can accurately clone the reference tone color and generate speech in multiple languages and accents.
+
+**2. Flexible Voice Style Control.**
+OpenVoice enables granular control over voice styles, such as emotion and accent, as well as other style parameters including rhythm, pauses, and intonation. 
+
+**3. Zero-shot Cross-lingual Voice Cloning.**
+Neither of the language of the generated speech nor the language of the reference speech needs to be presented in the massive-speaker multi-lingual training dataset.
+
+### OpenVoice V2
+
+In April 2024, we released OpenVoice V2, which includes all features in V1 and has:
+
+**1. Better Audio Quality.**
+OpenVoice V2 adopts a different training strategy that delivers better audio quality.
+
+**2. Native Multi-lingual Support.**
+English, Spanish, French, Chinese, Japanese and Korean are natively supported in OpenVoice V2.
+
+**3. Free Commercial Use.**
+Starting from April 2024, both V2 and V1 are released under MIT License. Free for commercial use.
+
+[Video](https://github.com/myshell-ai/OpenVoice/assets/40556743/3cba936f-82bf-476c-9e52-09f0f417bb2f)
+
+OpenVoice has been powering the instant voice cloning capability of [myshell.ai](https://app.myshell.ai/explore) since May 2023. Until Nov 2023, the voice cloning model has been used tens of millions of times by users worldwide, and witnessed the explosive user growth on the platform.
+
+## Main Contributors
+
+- [Zengyi Qin](https://www.qinzy.tech) at MIT
+- [Wenliang Zhao](https://wl-zhao.github.io) at Tsinghua University
+- [Xumin Yu](https://yuxumin.github.io) at Tsinghua University
+- [Ethan Sun](https://twitter.com/ethan_myshell) at MyShell
+
+## How to Use
+Please see [usage](docs/USAGE.md) for detailed instructions.
+
+## Common Issues
+
+Please see [QA](docs/QA.md) for common questions and answers. We will regularly update the question and answer list.
+
+## Citation
+```
+@article{qin2023openvoice,
+  title={OpenVoice: Versatile Instant Voice Cloning},
+  author={Qin, Zengyi and Zhao, Wenliang and Yu, Xumin and Sun, Xin},
+  journal={arXiv preprint arXiv:2312.01479},
+  year={2023}
+}
+```
+
+## License
+OpenVoice V1 and V2 are MIT Licensed. Free for both commercial and research use.
+
+## Acknowledgements
+This implementation is based on several excellent projects, [TTS](https://github.com/coqui-ai/TTS), [VITS](https://github.com/jaywalnut310/vits), and [VITS2](https://github.com/daniilrobnikov/vits2). Thanks for their awesome work!

demo_part1.ipynb

@@ -0,0 +1,236 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "b6ee1ede",
+   "metadata": {},
+   "source": [
+    "## Voice Style Control Demo"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "b7f043ee",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import os\n",
+    "import torch\n",
+    "from openvoice import se_extractor\n",
+    "from openvoice.api import BaseSpeakerTTS, ToneColorConverter"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "15116b59",
+   "metadata": {},
+   "source": [
+    "### Initialization"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "aacad912",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "ckpt_base = 'checkpoints/base_speakers/EN'\n",
+    "ckpt_converter = 'checkpoints/converter'\n",
+    "device=\"cuda:0\" if torch.cuda.is_available() else \"cpu\"\n",
+    "output_dir = 'outputs'\n",
+    "\n",
+    "base_speaker_tts = BaseSpeakerTTS(f'{ckpt_base}/config.json', device=device)\n",
+    "base_speaker_tts.load_ckpt(f'{ckpt_base}/checkpoint.pth')\n",
+    "\n",
+    "tone_color_converter = ToneColorConverter(f'{ckpt_converter}/config.json', device=device)\n",
+    "tone_color_converter.load_ckpt(f'{ckpt_converter}/checkpoint.pth')\n",
+    "\n",
+    "os.makedirs(output_dir, exist_ok=True)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "7f67740c",
+   "metadata": {},
+   "source": [
+    "### Obtain Tone Color Embedding"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "f8add279",
+   "metadata": {},
+   "source": [
+    "The `source_se` is the tone color embedding of the base speaker. \n",
+    "It is an average of multiple sentences generated by the base speaker. We directly provide the result here but\n",
+    "the readers feel free to extract `source_se` by themselves."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "63ff6273",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "source_se = torch.load(f'{ckpt_base}/en_default_se.pth').to(device)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "4f71fcc3",
+   "metadata": {},
+   "source": [
+    "The `reference_speaker.mp3` below points to the short audio clip of the reference whose voice we want to clone. We provide an example here. If you use your own reference speakers, please **make sure each speaker has a unique filename.** The `se_extractor` will save the `targeted_se` using the filename of the audio and **will not automatically overwrite.**"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "55105eae",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "reference_speaker = 'resources/example_reference.mp3' # This is the voice you want to clone\n",
+    "target_se, audio_name = se_extractor.get_se(reference_speaker, tone_color_converter, target_dir='processed', vad=True)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "a40284aa",
+   "metadata": {},
+   "source": [
+    "### Inference"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "73dc1259",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "save_path = f'{output_dir}/output_en_default.wav'\n",
+    "\n",
+    "# Run the base speaker tts\n",
+    "text = \"This audio is generated by OpenVoice.\"\n",
+    "src_path = f'{output_dir}/tmp.wav'\n",
+    "base_speaker_tts.tts(text, src_path, speaker='default', language='English', speed=1.0)\n",
+    "\n",
+    "# Run the tone color converter\n",
+    "encode_message = \"@MyShell\"\n",
+    "tone_color_converter.convert(\n",
+    "    audio_src_path=src_path, \n",
+    "    src_se=source_se, \n",
+    "    tgt_se=target_se, \n",
+    "    output_path=save_path,\n",
+    "    message=encode_message)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "6e3ea28a",
+   "metadata": {},
+   "source": [
+    "**Try with different styles and speed.** The style can be controlled by the `speaker` parameter in the `base_speaker_tts.tts` method. Available choices: friendly, cheerful, excited, sad, angry, terrified, shouting, whispering. Note that the tone color embedding need to be updated. The speed can be controlled by the `speed` parameter. Let's try whispering with speed 0.9."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "fd022d38",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "source_se = torch.load(f'{ckpt_base}/en_style_se.pth').to(device)\n",
+    "save_path = f'{output_dir}/output_whispering.wav'\n",
+    "\n",
+    "# Run the base speaker tts\n",
+    "text = \"This audio is generated by OpenVoice.\"\n",
+    "src_path = f'{output_dir}/tmp.wav'\n",
+    "base_speaker_tts.tts(text, src_path, speaker='whispering', language='English', speed=0.9)\n",
+    "\n",
+    "# Run the tone color converter\n",
+    "encode_message = \"@MyShell\"\n",
+    "tone_color_converter.convert(\n",
+    "    audio_src_path=src_path, \n",
+    "    src_se=source_se, \n",
+    "    tgt_se=target_se, \n",
+    "    output_path=save_path,\n",
+    "    message=encode_message)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "5fcfc70b",
+   "metadata": {},
+   "source": [
+    "**Try with different languages.** OpenVoice can achieve multi-lingual voice cloning by simply replace the base speaker. We provide an example with a Chinese base speaker here and we encourage the readers to try `demo_part2.ipynb` for a detailed demo."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "a71d1387",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "\n",
+    "ckpt_base = 'checkpoints/base_speakers/ZH'\n",
+    "base_speaker_tts = BaseSpeakerTTS(f'{ckpt_base}/config.json', device=device)\n",
+    "base_speaker_tts.load_ckpt(f'{ckpt_base}/checkpoint.pth')\n",
+    "\n",
+    "source_se = torch.load(f'{ckpt_base}/zh_default_se.pth').to(device)\n",
+    "save_path = f'{output_dir}/output_chinese.wav'\n",
+    "\n",
+    "# Run the base speaker tts\n",
+    "text = \"今天天气真好，我们一起出去吃饭吧。\"\n",
+    "src_path = f'{output_dir}/tmp.wav'\n",
+    "base_speaker_tts.tts(text, src_path, speaker='default', language='Chinese', speed=1.0)\n",
+    "\n",
+    "# Run the tone color converter\n",
+    "encode_message = \"@MyShell\"\n",
+    "tone_color_converter.convert(\n",
+    "    audio_src_path=src_path, \n",
+    "    src_se=source_se, \n",
+    "    tgt_se=target_se, \n",
+    "    output_path=save_path,\n",
+    "    message=encode_message)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "8e513094",
+   "metadata": {},
+   "source": [
+    "**Tech for good.** For people who will deploy OpenVoice for public usage: We offer you the option to add watermark to avoid potential misuse. Please see the ToneColorConverter class. **MyShell reserves the ability to detect whether an audio is generated by OpenVoice**, no matter whether the watermark is added or not."
+   ]
+  }
+ ],
+ "metadata": {
+  "interpreter": {
+   "hash": "9d70c38e1c0b038dbdffdaa4f8bfa1f6767c43760905c87a9fbe7800d18c6c35"
+  },
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.18"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

demo_part2.ipynb

@@ -0,0 +1,195 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "b6ee1ede",
+   "metadata": {},
+   "source": [
+    "## Cross-Lingual Voice Clone Demo"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "b7f043ee",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import os\n",
+    "import torch\n",
+    "from openvoice import se_extractor\n",
+    "from openvoice.api import ToneColorConverter"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "15116b59",
+   "metadata": {},
+   "source": [
+    "### Initialization"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "aacad912",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "ckpt_converter = 'checkpoints/converter'\n",
+    "device=\"cuda:0\" if torch.cuda.is_available() else \"cpu\"\n",
+    "output_dir = 'outputs'\n",
+    "\n",
+    "tone_color_converter = ToneColorConverter(f'{ckpt_converter}/config.json', device=device)\n",
+    "tone_color_converter.load_ckpt(f'{ckpt_converter}/checkpoint.pth')\n",
+    "\n",
+    "os.makedirs(output_dir, exist_ok=True)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "3db80fcf",
+   "metadata": {},
+   "source": [
+    "In this demo, we will use OpenAI TTS as the base speaker to produce multi-lingual speech audio. The users can flexibly change the base speaker according to their own needs. Please create a file named `.env` and place OpenAI key as `OPENAI_API_KEY=xxx`. We have also provided a Chinese base speaker model (see `demo_part1.ipynb`)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "3b245ca3",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from openai import OpenAI\n",
+    "from dotenv import load_dotenv\n",
+    "\n",
+    "# Please create a file named .env and place your\n",
+    "# OpenAI key as OPENAI_API_KEY=xxx\n",
+    "load_dotenv() \n",
+    "\n",
+    "client = OpenAI(api_key=os.environ.get(\"OPENAI_API_KEY\"))\n",
+    "\n",
+    "response = client.audio.speech.create(\n",
+    "    model=\"tts-1\",\n",
+    "    voice=\"nova\",\n",
+    "    input=\"This audio will be used to extract the base speaker tone color embedding. \" + \\\n",
+    "        \"Typically a very short audio should be sufficient, but increasing the audio \" + \\\n",
+    "        \"length will also improve the output audio quality.\"\n",
+    ")\n",
+    "\n",
+    "response.stream_to_file(f\"{output_dir}/openai_source_output.mp3\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "7f67740c",
+   "metadata": {},
+   "source": [
+    "### Obtain Tone Color Embedding"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "f8add279",
+   "metadata": {},
+   "source": [
+    "The `source_se` is the tone color embedding of the base speaker. \n",
+    "It is an average for multiple sentences with multiple emotions\n",
+    "of the base speaker. We directly provide the result here but\n",
+    "the readers feel free to extract `source_se` by themselves."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "63ff6273",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "base_speaker = f\"{output_dir}/openai_source_output.mp3\"\n",
+    "source_se, audio_name = se_extractor.get_se(base_speaker, tone_color_converter, vad=True)\n",
+    "\n",
+    "reference_speaker = 'resources/example_reference.mp3' # This is the voice you want to clone\n",
+    "target_se, audio_name = se_extractor.get_se(reference_speaker, tone_color_converter, vad=True)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "a40284aa",
+   "metadata": {},
+   "source": [
+    "### Inference"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "73dc1259",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Run the base speaker tts\n",
+    "text = [\n",
+    "    \"MyShell is a decentralized and comprehensive platform for discovering, creating, and staking AI-native apps.\",\n",
+    "    \"MyShell es una plataforma descentralizada y completa para descubrir, crear y apostar por aplicaciones nativas de IA.\",\n",
+    "    \"MyShell est une plateforme décentralisée et complète pour découvrir, créer et miser sur des applications natives d'IA.\",\n",
+    "    \"MyShell ist eine dezentralisierte und umfassende Plattform zum Entdecken, Erstellen und Staken von KI-nativen Apps.\",\n",
+    "    \"MyShell è una piattaforma decentralizzata e completa per scoprire, creare e scommettere su app native di intelligenza artificiale.\",\n",
+    "    \"MyShellは、AIネイティブアプリの発見、作成、およびステーキングのための分散型かつ包括的なプラットフォームです。\",\n",
+    "    \"MyShell — это децентрализованная и всеобъемлющая платформа для обнаружения, создания и стейкинга AI-ориентированных приложений.\",\n",
+    "    \"MyShell هي منصة لامركزية وشاملة لاكتشاف وإنشاء ورهان تطبيقات الذكاء الاصطناعي الأصلية.\",\n",
+    "    \"MyShell是一个去中心化且全面的平台，用于发现、创建和投资AI原生应用程序。\",\n",
+    "    \"MyShell एक विकेंद्रीकृत और व्यापक मंच है, जो AI-मूल ऐप्स की खोज, सृजन और स्टेकिंग के लिए है।\",\n",
+    "    \"MyShell é uma plataforma descentralizada e abrangente para descobrir, criar e apostar em aplicativos nativos de IA.\"\n",
+    "]\n",
+    "src_path = f'{output_dir}/tmp.wav'\n",
+    "\n",
+    "for i, t in enumerate(text):\n",
+    "\n",
+    "    response = client.audio.speech.create(\n",
+    "        model=\"tts-1\",\n",
+    "        voice=\"nova\",\n",
+    "        input=t,\n",
+    "    )\n",
+    "\n",
+    "    response.stream_to_file(src_path)\n",
+    "\n",
+    "    save_path = f'{output_dir}/output_crosslingual_{i}.wav'\n",
+    "\n",
+    "    # Run the tone color converter\n",
+    "    encode_message = \"@MyShell\"\n",
+    "    tone_color_converter.convert(\n",
+    "        audio_src_path=src_path, \n",
+    "        src_se=source_se, \n",
+    "        tgt_se=target_se, \n",
+    "        output_path=save_path,\n",
+    "        message=encode_message)"
+   ]
+  }
+ ],
+ "metadata": {
+  "interpreter": {
+   "hash": "9d70c38e1c0b038dbdffdaa4f8bfa1f6767c43760905c87a9fbe7800d18c6c35"
+  },
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.18"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

demo_part3.ipynb

@@ -0,0 +1,145 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Multi-Accent and Multi-Lingual Voice Clone Demo with MeloTTS"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import os\n",
+    "import torch\n",
+    "from openvoice import se_extractor\n",
+    "from openvoice.api import ToneColorConverter"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Initialization\n",
+    "\n",
+    "In this example, we will use the checkpoints from OpenVoiceV2. OpenVoiceV2 is trained with more aggressive augmentations and thus demonstrate better robustness in some cases."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "ckpt_converter = 'checkpoints_v2/converter'\n",
+    "device = \"cuda:0\" if torch.cuda.is_available() else \"cpu\"\n",
+    "output_dir = 'outputs_v2'\n",
+    "\n",
+    "tone_color_converter = ToneColorConverter(f'{ckpt_converter}/config.json', device=device)\n",
+    "tone_color_converter.load_ckpt(f'{ckpt_converter}/checkpoint.pth')\n",
+    "\n",
+    "os.makedirs(output_dir, exist_ok=True)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Obtain Tone Color Embedding\n",
+    "We only extract the tone color embedding for the target speaker. The source tone color embeddings can be directly loaded from `checkpoints_v2/ses` folder."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "\n",
+    "reference_speaker = 'resources/example_reference.mp3' # This is the voice you want to clone\n",
+    "target_se, audio_name = se_extractor.get_se(reference_speaker, tone_color_converter, vad=True)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "#### Use MeloTTS as Base Speakers\n",
+    "\n",
+    "MeloTTS is a high-quality multi-lingual text-to-speech library by @MyShell.ai, supporting languages including English (American, British, Indian, Australian, Default), Spanish, French, Chinese, Japanese, Korean. In the following example, we will use the models in MeloTTS as the base speakers. "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from melo.api import TTS\n",
+    "\n",
+    "texts = {\n",
+    "    'EN_NEWEST': \"Did you ever hear a folk tale about a giant turtle?\",  # The newest English base speaker model\n",
+    "    'EN': \"Did you ever hear a folk tale about a giant turtle?\",\n",
+    "    'ES': \"El resplandor del sol acaricia las olas, pintando el cielo con una paleta deslumbrante.\",\n",
+    "    'FR': \"La lueur dorée du soleil caresse les vagues, peignant le ciel d'une palette éblouissante.\",\n",
+    "    'ZH': \"在这次vacation中，我们计划去Paris欣赏埃菲尔铁塔和卢浮宫的美景。\",\n",
+    "    'JP': \"彼は毎朝ジョギングをして体を健康に保っています。\",\n",
+    "    'KR': \"안녕하세요! 오늘은 날씨가 정말 좋네요.\",\n",
+    "}\n",
+    "\n",
+    "\n",
+    "src_path = f'{output_dir}/tmp.wav'\n",
+    "\n",
+    "# Speed is adjustable\n",
+    "speed = 1.0\n",
+    "\n",
+    "for language, text in texts.items():\n",
+    "    model = TTS(language=language, device=device)\n",
+    "    speaker_ids = model.hps.data.spk2id\n",
+    "    \n",
+    "    for speaker_key in speaker_ids.keys():\n",
+    "        speaker_id = speaker_ids[speaker_key]\n",
+    "        speaker_key = speaker_key.lower().replace('_', '-')\n",
+    "        \n",
+    "        source_se = torch.load(f'checkpoints_v2/base_speakers/ses/{speaker_key}.pth', map_location=device)\n",
+    "        if torch.backends.mps.is_available() and device == 'cpu':\n",
+    "            torch.backends.mps.is_available = lambda: False\n",
+    "        model.tts_to_file(text, speaker_id, src_path, speed=speed)\n",
+    "        save_path = f'{output_dir}/output_v2_{speaker_key}.wav'\n",
+    "\n",
+    "        # Run the tone color converter\n",
+    "        encode_message = \"@MyShell\"\n",
+    "        tone_color_converter.convert(\n",
+    "            audio_src_path=src_path, \n",
+    "            src_se=source_se, \n",
+    "            tgt_se=target_se, \n",
+    "            output_path=save_path,\n",
+    "            message=encode_message)"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "melo",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.18"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

requirements.txt

@@ -0,0 +1,16 @@
+librosa==0.9.1
+faster-whisper==0.9.0
+pydub==0.25.1
+wavmark==0.0.3
+numpy==1.22.0
+eng_to_ipa==0.0.2
+inflect==7.0.0
+unidecode==1.3.7
+whisper-timestamped==1.14.2
+openai
+python-dotenv
+pypinyin==0.50.0
+cn2an==0.5.22
+jieba==0.42.1
+gradio==3.48.0
+langid==1.1.6

setup.py

@@ -0,0 +1,45 @@
+from setuptools import setup, find_packages
+
+
+setup(name='MyShell-OpenVoice',
+      version='0.0.0',
+      description='Instant voice cloning by MyShell.',
+      long_description=open('README.md').read().strip(),
+      long_description_content_type='text/markdown',
+      keywords=[
+            'text-to-speech',
+            'tts',
+            'voice-clone',
+            'zero-shot-tts'
+      ],
+      url='https://github.com/myshell-ai/OpenVoice',
+      project_urls={
+        'Documentation': 'https://github.com/myshell-ai/OpenVoice/blob/main/docs/USAGE.md',
+        'Changes': 'https://github.com/myshell-ai/OpenVoice/releases',
+        'Code': 'https://github.com/myshell-ai/OpenVoice',
+        'Issue tracker': 'https://github.com/myshell-ai/OpenVoice/issues',
+      },
+      author='MyShell',
+      author_email='[email protected]',
+      license='MIT License',
+      packages=find_packages(),
+
+      python_requires='>=3.9',
+      install_requires=[
+            'librosa==0.9.1',
+            'faster-whisper==0.9.0',
+            'pydub==0.25.1',
+            'wavmark==0.0.3',
+            'numpy==1.22.0',
+            'eng_to_ipa==0.0.2',
+            'inflect==7.0.0',
+            'unidecode==1.3.7',
+            'whisper-timestamped==1.14.2',
+            'pypinyin==0.50.0',
+            'cn2an==0.5.22',
+            'jieba==0.42.1',
+            'gradio==3.48.0',
+            'langid==1.1.6'
+      ],
+      zip_safe=False
+      )