AI-CosyVoice

我是通义实验室语音团队全新推出的生成式语音大模型，提供舒适自然的语音合成能力。

安装

1. 在PyCharm IDEA中拉取代码git@github.com:FunAudioLLM/CosyVoice.git

2. 在PyCharm IDEA中的终端里面拉取子项目，在终端里面执行git submodule update --init --recursive

3. 下载安装Miniconda，选择下载Miniconda Installers，默认安装即可。

4. 在PyCharm IDEA配置Conda，在Settings->Project:CosyVoice->Python Interpreter中点击Add Interpreter->Add Local Interpreter界面里面填入

   Type: Conda
   Python: 3.12
   Name: CosyVoice
   Path to conda: C:\ProgramData\Miniconda3\condabin\conda.bat

5. 在PyCharm IDEA中点击左下角的Terminal，然后再窗口标签的+旁，点击·=下拉箭头，选择Command Prompt终端，注意括号里面要是CosyVoice，不然会将包安装到系统环境。例如：(CosyVoice) D:\workspace\CosyVoice>

6. 安装库，在Command Prompt终端中执行pip install -r requirements.txt -i https://mirrors.aliyun.com/pypi/simple/ --trusted-host=mirrors.aliyun.com

7. 安装FFmpeg，在Command Prompt终端中执行conda install ffmpeg -c conda-forge

   #额外非必要步骤，只是这个工具可以从视频提取音频，可以拿到后面的素材模板
   ffmpeg -i input.mkv -q:a 0 -map a output.mp3

8. 下载大模型，在PyCharm IDEA中新建download_model.py，然后在文件上右键运行RUN 'download_model'

   from modelscope import snapshot_download
   
   model_dir = snapshot_download(
       'iic/CosyVoice2-0.5B',
       local_dir='pretrained_models/CosyVoice2-0.5B'
   )
   
   print(f'Model downloaded to: {model_dir}')

9. 在PyCharm IDEA打开项目根目录中的webui.py文件，在代码中找到if __name__ == '__main__':这一行，然后点击前面的运行符号。

10. 运行成功后，访问http://localhost:8000/，然后设置如下参数，点击`生成音频`

    输入合成文本：要转换音频的文字
    选择推理模式：3s极速复刻
    选择prompt音频文件：上传要模仿的样本音频
    输入prompt文本：样本音频对应的文字

部署

# 创建python环境
conda create -n CosyVoice630 python=3.10 -y
# 激活
conda activate CosyVoice630
# 下载依赖
pip install -r requirements.txt -i https://mirrors.aliyun.com/pypi/simple/ --trusted-host=mirrors.aliyun.com
# 克隆
conda create -n CosyVoice630_vllm --clone CosyVoice630
# 激活新的
conda activate CosyVoice630_vllm
# 安装vllm
pip install vllm==v0.9.0 -i https://mirrors.aliyun.com/pypi/simple/ --trusted-host=mirrors.aliyun.com
python vllm_example.py

环境准备

1. 禁用 nouveau 驱动（NVIDIA 官方推荐）

   # 查看 nouveau 是否加载
   lsmod | grep nouveau
   lsmod | grep nvidia #检查是否有老版本
   # 有卸载
   sudo /usr/bin/nvidia-uninstall
   sudo rm -rf /usr/local/cuda*
   sudo rm -rf /lib/modules/$(uname -r)/kernel/drivers/video/nvidia*
   sudo rm -rf /etc/modprobe.d/nvidia*
   # 黑名单禁用 nouveau 驱动（两条配置）
   sudo bash -c "echo 'blacklist nouveau' > /etc/modprobe.d/blacklist-nouveau.conf"
   sudo bash -c "echo 'options nouveau modeset=0' >> /etc/modprobe.d/blacklist-nouveau.conf"
   # 查看确认配置
   cat /etc/modprobe.d/blacklist-nouveau.conf
   # 重建 initramfs
   sudo dracut --force
   # 再次检查 nouveau 是否加载
   lsmod | grep nouveau
   # 重启系统生效
   sudo reboot

2. 安装 CUDA 12.1

   # 下载 CUDA 12.1 安装包
   wget https://developer.download.nvidia.com/compute/cuda/12.1.1/local_installers/cuda_12.1.1_530.30.02_linux.run
   # 运行安装包（根据提示完成安装）
   sudo sh cuda_12.1.1_530.30.02_linux.run
   # 验证 CUDA 安装
   nvidia-smi
   # 添加 CUDA bin 到 PATH
   echo 'export PATH=/usr/local/cuda-12.1/bin:$PATH' >> ~/.bashrc
   # 添加 CUDA lib64 到 LD_LIBRARY_PATH
   echo 'export LD_LIBRARY_PATH=/usr/local/cuda-12.1/lib64:$LD_LIBRARY_PATH' >> ~/.bashrc
   # 使配置立即生效
   source ~/.bashrc
   # 验证 nvcc
   nvcc --version

3. 安装 Miniconda（Python 发行版）

   # 下载 Miniconda 安装脚本
   wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
   # 执行安装脚本（按提示操作）
   bash Miniconda3-latest-Linux-x86_64.sh
   # 初始化 conda 环境变量
   source /root/miniconda3/etc/profile.d/conda.sh
   # 配置envs目录，防止根目录爆满
   vi /etc/bashrc
   # 末尾添加
   export CONDA_ENVS_PATH=/data/envs
   # 立即生效
   source /etc/bashrc

4. 安装环境依赖

   dnf update   #apt-get update
   dnf install sox sox-devel -y #apt-get install sox libsox-dev -y

常见问题

1. 安装依赖时WeTextProcessing安装失败提示pynini错误时

   解决：执行conda install -c conda-forge pynini=2.1.5，版本号可以再错误日志里面查看

   原因：用 conda 安装的这种包一般是编译好的二进制包，不需要自己编译，避免了你之前 pip 安装时遇到的编译错误

   详细错误信息：

   (ai-tts) D:\workspace\ai-tts>pip install WeTextProcessing==1.0.3
   Collecting WeTextProcessing==1.0.3
     Downloading WeTextProcessing-1.0.3-py3-none-any.whl.metadata (7.2 kB)
   Collecting pynini==2.1.5 (from WeTextProcessing==1.0.3)
     Downloading pynini-2.1.5.tar.gz (627 kB)
        ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 627.6/627.6 kB 106.3 kB/s eta 0:00:00
     Preparing metadata (setup.py) ... done
   Collecting importlib-resources (from WeTextProcessing==1.0.3)
     Downloading importlib_resources-6.5.2-py3-none-any.whl.metadata (3.9 kB)
   Requirement already satisfied: Cython>=0.29 in c:\users\lenovo\.conda\envs\ai-tts\lib\site-packages (from pynini==2.1.5->WeTextProcessing==1.0.3) (3.1.2)
   Downloading WeTextProcessing-1.0.3-py3-none-any.whl (2.0 MB)
      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.0/2.0 MB 58.2 kB/s eta 0:00:00
   Downloading importlib_resources-6.5.2-py3-none-any.whl (37 kB)
   Building wheels for collected packages: pynini
     DEPRECATION: Building 'pynini' using the legacy setup.py bdist_wheel mechanism, which will be removed in a future version. pip 25.3 will enforce this behaviour
    change. A possible replacement is to use the standardized build interface by setting the --use-pep517 option, (possibly combined with --no-build-isolation), or adding a pyproject.toml file to the source tree of 'pynini'. Discussion can be found at https://github.com/pypa/pip/issues/6334
     Building wheel for pynini (setup.py) ... error
     error: subprocess-exited-with-error
   
     × python setup.py bdist_wheel did not run successfully.
     │ exit code: 1
     ╰─> [59 lines of output]
         C:\Users\Lenovo\.conda\envs\ai-tts\lib\site-packages\setuptools\dist.py:759: SetuptoolsDeprecationWarning: License classifiers are deprecated.
         !!
   
                 ********************************************************************************
                 Please consider removing the following classifiers in favor of a SPDX license expression:
   
                 License :: OSI Approved :: Apache Software License
   
                 See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details.
                 ********************************************************************************
   
         !!
           self._finalize_license_expression()
         running bdist_wheel
         running build
         running build_py
         creating build\lib.win-amd64-cpython-310\pynini
         copying pynini\__init__.py -> build\lib.win-amd64-cpython-310\pynini
         creating build\lib.win-amd64-cpython-310\pywrapfst
         copying pywrapfst\__init__.py -> build\lib.win-amd64-cpython-310\pywrapfst

2. 因网络原因安装conda install -c conda-forge pynini=2.1.5失败

   # ---有外网的环境-----------
   # 模拟相同的python环境
   conda create -n ai-tts python=3.10 -y
   conda activate ai-tts
   # 执行无外网环境安装不了的包（只区分liunx x86还是arm，不区分liunx具体系统及版本）
   conda install -c conda-forge pynini=2.1.5
   # 创建离线包的目录
   mkdir /mnt/d/ai/offline-env
   # 导出离线包的下载地址
   conda list --explicit > /mnt/d/ai/offline-env/ai-tts.txt
   cd offline-env/
   # 下载
   wget -i ai-tts.txt
   # 压缩离线包
   tar czvf offline-env.tar.gz offline-env/ 
   # --------------无外网环境------------------
   # 解压离线包
   tar -xzf offline-env.tar.gz
   cd offline-env/
   # 进入离线目录，离线安装
   conda install --offline --use-local *.tar.bz2

参考：

https://doupoa.site/archives/581