Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multivocoder Update: A substantial infrastructure upgrade proposed as a version improvement. #752

Draft
wants to merge 86 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
86 commits
Select commit Hold shift + click to select a range
e9e70a1
Add MultiVocoders
Vidalnt Jul 7, 2024
54c4eb1
Add Config ( Voc_Type )
Vidalnt Jul 10, 2024
eae13fc
Temp Fix
Vidalnt Jul 11, 2024
1ba64ee
Fix.
Vidalnt Jul 11, 2024
f0adc6f
fix missmatch
blaisewf Jul 11, 2024
067c376
rename vocoder_type + organize better encoders
blaisewf Jul 11, 2024
2f9d7da
organize ffn function
blaisewf Jul 11, 2024
9aec49e
fix encoders
blaisewf Jul 11, 2024
31f9181
fix text_enc_hidden_dim on configs
blaisewf Jul 13, 2024
4f536e8
fix encoder
blaisewf Jul 14, 2024
87e14ec
minor change on encoder + remove some docstrings
blaisewf Jul 14, 2024
5c124f9
Remove Things
Vidalnt Jul 18, 2024
a44008c
Fix.
Vidalnt Jul 19, 2024
47ef748
Add BigvGan V2 And Mel
Vidalnt Jul 20, 2024
0e1181e
Update requirements.txt
Vidalnt Jul 20, 2024
ce9fb3c
Merge branch 'main' into multivoc
blaisewf Jul 30, 2024
a75b5bb
fix core
blaisewf Jul 30, 2024
4d644f5
fix save model
blaisewf Jul 30, 2024
34211d8
Merge branch 'main' into multivoc
blaisewf Jul 30, 2024
ee7c2db
Merge branch 'main' into multivoc
blaisewf Jul 30, 2024
a4d0434
Add San Discriminators
Vidalnt Jul 31, 2024
c343eb8
Add Bivgsan
Vidalnt Jul 31, 2024
b54a6a2
Add Config Mpd
Vidalnt Jul 31, 2024
f96faf8
Remove Values Default
Vidalnt Jul 31, 2024
6c4f881
remove param on extract
blaisewf Jul 31, 2024
eac7d80
fix gen config
blaisewf Jul 31, 2024
f136095
rename sampling_rate to sample_rate
blaisewf Jul 31, 2024
8083030
Fixed
Vidalnt Jul 31, 2024
3d1f43e
configs refactor
blaisewf Jul 31, 2024
a6832f3
Fix.
Vidalnt Jul 31, 2024
dd1133d
full path imports
blaisewf Jul 31, 2024
1c7a439
Merge branch 'multivoc' of https://github.com/IAHispano/Applio into m…
blaisewf Jul 31, 2024
7b1bcd0
refactor + convert to half
blaisewf Aug 1, 2024
ecfeac3
fix training
blaisewf Aug 1, 2024
f664303
Merge branch 'main' into multivoc
blaisewf Aug 2, 2024
805d1ee
fix arg
blaisewf Aug 2, 2024
4f15132
Organising discriminators
Vidalnt Aug 2, 2024
9e2d8c2
Fix Imports
Vidalnt Aug 2, 2024
a87228b
Merge branch 'main' into multivoc
blaisewf Aug 2, 2024
57d3baf
Organize better all the modules
blaisewf Aug 3, 2024
3437c6d
Simplify os.getcwd()
blaisewf Aug 3, 2024
3a2ead9
Optimising code
Vidalnt Aug 3, 2024
2e8da13
improve all paths
blaisewf Aug 4, 2024
4e5c539
improve loadthemes system
blaisewf Aug 4, 2024
21de447
Fix
Vidalnt Aug 5, 2024
0cdf652
Merge main into multivoc
blaisewf Aug 5, 2024
a8fbfc3
Fix Regex
Vidalnt Aug 5, 2024
080692f
Merge branch 'main' into multivoc
blaisewf Aug 5, 2024
fb6d52b
Merge branch 'main' into multivoc
blaisewf Aug 6, 2024
d88ed4c
Fix
Vidalnt Aug 7, 2024
47ab1dd
Temp Fix
Vidalnt Aug 7, 2024
4be99bc
Temp Print
Vidalnt Aug 7, 2024
95231b9
Temp Print
Vidalnt Aug 8, 2024
a5708a7
Temp Print
Vidalnt Aug 8, 2024
2148bac
Temp Fix [train.py]
Vidalnt Aug 8, 2024
1ea1e32
Fix Args Discriminators
Vidalnt Aug 8, 2024
3e76c47
Fix Args
Vidalnt Aug 8, 2024
d1b6def
Fix Value Arg
Vidalnt Aug 8, 2024
f99ca50
Fix Discriminators
Vidalnt Aug 8, 2024
8d860f6
Fix
Vidalnt Aug 8, 2024
079f71f
Fix Period Arg
Vidalnt Aug 8, 2024
4a1d051
Fix
Vidalnt Aug 8, 2024
834c30b
Fix Args
Vidalnt Aug 8, 2024
32e7bf0
Fix Name Bigvsan
Vidalnt Aug 8, 2024
dee5aaf
Uh.
Vidalnt Aug 8, 2024
0db9fd3
Remove Log ( nnAudio )
Vidalnt Aug 8, 2024
063fbd6
Fix "is_san"
Vidalnt Aug 8, 2024
e08ab2c
Debug -> Error
Vidalnt Aug 8, 2024
ba5d2c1
Removing already defined arguments
Vidalnt Aug 8, 2024
cdd7a95
Removing already defined arguments
Vidalnt Aug 8, 2024
d0faa32
Replace is_san to flg_train
Vidalnt Aug 8, 2024
e55434e
Add San Loss
Vidalnt Aug 8, 2024
3803629
Fix Combined
Vidalnt Aug 8, 2024
ff45ece
Fix
Vidalnt Aug 8, 2024
24a1590
Merge branch 'main' into multivoc
blaisewf Aug 12, 2024
90dd735
Minify code
Vidalnt Aug 14, 2024
64f6ad0
Merge branch 'main' into multivoc
Vidalnt Aug 19, 2024
10c2a8d
Fix Merge
Vidalnt Aug 19, 2024
410a858
Fix Extract
Vidalnt Aug 24, 2024
7558401
Missing Value
Vidalnt Aug 24, 2024
674e920
Clean Merge
Vidalnt Aug 24, 2024
14e3f97
Merge branch 'main' into multivoc
Vidalnt Aug 25, 2024
5110d5e
Merge branch 'main' into exp/multivocoder
Vidalnt Aug 30, 2024
3854124
Merge branch 'main' into exp/multivocoder
Vidalnt Oct 8, 2024
3166dcf
Update train.py
Vidalnt Oct 12, 2024
b5bea7d
fix: :bug: Merge
Vidalnt Oct 12, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
File renamed without changes.
File renamed without changes.
37 changes: 37 additions & 0 deletions .github/PULL_REQUEST_TEMPLATE/pull_request.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
<!--- Provide a general summary of your changes in the Title above -->

## Description

<!--- Describe your changes in detail -->

## Motivation and Context

<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here. -->

## How has this been tested?

<!--- Please describe in detail how you tested your changes. -->
<!--- Include details of your testing environment, tests ran to see how -->
<!--- your change affects other areas of the code, etc. -->

## Screenshots (if appropriate):

## Types of changes

<!--- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->

- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

## Checklist:

<!--- Go over all the following points, and put an `x` in all the boxes that apply. -->
<!--- If you're unsure about any of these, don't hesitate to ask. We're here to help! -->

- [ ] My code follows the code style of this project.
- [ ] My change requires a change to the documentation.
- [ ] I have updated the documentation accordingly.
- [ ] I have added tests to cover my changes.
- [ ] All new and existing tests passed.
20 changes: 20 additions & 0 deletions .github/release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# .github/release.yml

changelog:
exclude:
labels:
- ignore-for-release
authors:
- blaise.tk
categories:
- title: Breaking Changes 🛠
labels:
- Fix
- breaking-change
- title: Exciting New Features 🎉
labels:
- Add
- enhancement
- title: Other Changes
labels:
- "*"
3 changes: 1 addition & 2 deletions assets/flask/routes.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,7 @@
import signal
from flask import Flask, request, redirect

now_dir = os.getcwd()
sys.path.append(now_dir)
sys.path.append(os.getcwd())

from core import run_download_script

Expand Down
7 changes: 3 additions & 4 deletions assets/flask/server.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,9 @@
import json

# Constants
NOW_DIR = os.getcwd()
CONFIG_FILE = os.path.join(NOW_DIR, "assets", "config.json")
ENV_PATH = os.path.join(NOW_DIR, "env", "python.exe")
FLASK_SCRIPT_PATH = os.path.join(NOW_DIR, "assets", "flask", "routes.py")
CONFIG_FILE = os.path.join(os.getcwd(), "assets", "config.json")
ENV_PATH = os.path.join(os.getcwd(), "env", "python.exe")
FLASK_SCRIPT_PATH = os.path.join(os.getcwd(), "assets", "flask", "routes.py")
HOST = "localhost"
PORT = 8000
TIMEOUT = 2
Expand Down
8 changes: 4 additions & 4 deletions assets/formant_shift/f2m.json
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{
"formant_qfrency": 1.0,
"formant_timbre": 0.8
}
{
"formant_qfrency": 1.0,
"formant_timbre": 0.8
}
8 changes: 4 additions & 4 deletions assets/formant_shift/m2f.json
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{
"formant_qfrency": 1.0,
"formant_timbre": 1.2
}
{
"formant_qfrency": 1.0,
"formant_timbre": 1.2
}
8 changes: 4 additions & 4 deletions assets/formant_shift/random.json
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{
"formant_qfrency": 32.0,
"formant_timbre": 9.8
}
{
"formant_qfrency": 32.0,
"formant_timbre": 9.8
}
7 changes: 3 additions & 4 deletions assets/i18n/i18n.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,16 +3,15 @@
from pathlib import Path
from locale import getdefaultlocale

now_dir = os.getcwd()
sys.path.append(now_dir)
sys.path.append(os.getcwd())


class I18nAuto:
LANGUAGE_PATH = os.path.join(now_dir, "assets", "i18n", "languages")
LANGUAGE_PATH = os.path.join(os.getcwd(), "assets", "i18n", "languages")

def __init__(self, language=None):
with open(
os.path.join(now_dir, "assets", "config.json"), "r", encoding="utf8"
os.path.join(os.getcwd(), "assets", "config.json"), "r", encoding="utf8"
) as file:
config = json.load(file)
override = config["lang"]["override"]
Expand Down
614 changes: 307 additions & 307 deletions assets/i18n/languages/pt_BR.json

Large diffs are not rendered by default.

11 changes: 5 additions & 6 deletions assets/installation_checker.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
import sys
import os

now_dir = os.getcwd()
sys.path.append(now_dir)
sys.path.append(os.getcwd())


class InstallationError(Exception):
Expand All @@ -14,24 +13,24 @@ def __init__(self, message="InstallationError"):
def check_installation():
try:
system_drive = os.getenv("SystemDrive")
current_drive = os.path.splitdrive(now_dir)[0]
current_drive = os.path.splitdrive(os.getcwd())[0]
if current_drive.upper() != system_drive.upper():
raise InstallationError(
f"Installation Error: The current working directory is on drive {current_drive}, but the default system drive is {system_drive}. Please move Applio to the {system_drive} drive."
)
except:
pass
else:
if "OneDrive" in now_dir:
if "OneDrive" in os.getcwd():
raise InstallationError(
"Installation Error: The current working directory is located in OneDrive. Please move Applio to a different folder."
)
elif " " in now_dir:
elif " " in os.getcwd():
raise InstallationError(
"Installation Error: The current working directory contains spaces. Please move Applio to a folder without spaces in its path."
)
try:
now_dir.encode("ascii")
os.getcwd().encode("ascii")
except UnicodeEncodeError:
raise InstallationError(
"Installation Error: The current working directory contains non-ASCII characters. Please move Applio to a folder with only ASCII characters in its path."
Expand Down
102 changes: 42 additions & 60 deletions assets/themes/loadThemes.py
Original file line number Diff line number Diff line change
@@ -1,118 +1,100 @@
import json
import os
import sys
import importlib
import gradio as gr

now_dir = os.getcwd()

folder = os.path.dirname(os.path.abspath(__file__))
folder = os.path.dirname(folder)
folder = os.path.dirname(folder)
folder = os.path.join(folder, "assets", "themes")
config_file = os.path.join(now_dir, "assets", "config.json")

import sys
# Setting up the paths
base_folder = os.path.dirname(
os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
)
themes_folder = os.path.join(base_folder, "assets", "themes")
config_file_path = os.path.join(os.getcwd(), "assets", "config.json")

sys.path.append(folder)
sys.path.append(themes_folder)


def get_class(filename):
with open(filename, "r", encoding="utf8") as file:
for line_number, line in enumerate(file, start=1):
if "class " in line:
found = line.split("class ")[1].split(":")[0].split("(")[0].strip()
return found
break
for line in file:
if line.startswith("class "):
return line.split("class ")[1].split(":")[0].split("(")[0].strip()
return None


def get_list():

themes_from_files = [
os.path.splitext(name)[0]
for root, _, files in os.walk(folder, topdown=False)
for root, _, files in os.walk(themes_folder, topdown=False)
for name in files
if name.endswith(".py") and root == folder
if name.endswith(".py") and root == themes_folder
]

json_file_path = os.path.join(folder, "theme_list.json")
json_file_path = os.path.join(themes_folder, "theme_list.json")

try:
with open(json_file_path, "r", encoding="utf8") as json_file:
themes_from_url = [item["id"] for item in json.load(json_file)]
except FileNotFoundError:
themes_from_url = []

combined_themes = set(themes_from_files + themes_from_url)

return list(combined_themes)
return list(set(themes_from_files + themes_from_url))


def select_theme(name):
selected_file = name + ".py"
full_path = os.path.join(folder, selected_file)
selected_file = f"{name}.py"
full_path = os.path.join(themes_folder, selected_file)

if not os.path.exists(full_path):
with open(config_file, "r", encoding="utf8") as json_file:
try:
with open(config_file_path, "r", encoding="utf8") as json_file:
config_data = json.load(json_file)

config_data["theme"]["file"] = None
config_data["theme"]["class"] = name
if not os.path.exists(full_path):
config_data["theme"]["file"] = None
config_data["theme"]["class"] = name
else:
class_name = get_class(full_path)
if class_name:
config_data["theme"]["file"] = selected_file
config_data["theme"]["class"] = class_name
else:
print(f"Theme {name} was not found.")
return

with open(config_file, "w", encoding="utf8") as json_file:
with open(config_file_path, "w", encoding="utf8") as json_file:
json.dump(config_data, json_file, indent=2)
print(f"Theme {name} successfully selected, restart applio.")
gr.Info(f"Theme {name} successfully selected, restart applio.")
return

class_found = get_class(full_path)
if class_found:
with open(config_file, "r", encoding="utf8") as json_file:
config_data = json.load(json_file)

config_data["theme"]["file"] = selected_file
config_data["theme"]["class"] = class_found

with open(config_file, "w", encoding="utf8") as json_file:
json.dump(config_data, json_file, indent=2)
print(f"Theme {name} successfully selected, restart applio.")
gr.Info(f"Theme {name} successfully selected, restart applio.")
else:
print(f"Theme {name} was not found.")
print(f"Theme {name} successfully selected, restart Applio.")
gr.Info(f"Theme {name} successfully selected, restart Applio.")
except Exception as error:
print(f"An error occurred: {error}")


def read_json():
try:
with open(config_file, "r", encoding="utf8") as json_file:
with open(config_file_path, "r", encoding="utf8") as json_file:
data = json.load(json_file)
selected_file = data["theme"]["file"]
class_name = data["theme"]["class"]

if selected_file is not None and class_name:
return class_name
elif selected_file == None and class_name:
return class_name
else:
return "ParityError/Interstellar"
return data["theme"].get("class", "ParityError/Interstellar")
except Exception as error:
print(f"An error occurred loading the theme: {error}")
return "ParityError/Interstellar"


def load_json():
try:
with open(config_file, "r", encoding="utf8") as json_file:
with open(config_file_path, "r", encoding="utf8") as json_file:
data = json.load(json_file)
selected_file = data["theme"]["file"]
class_name = data["theme"]["class"]
selected_file = data["theme"].get("file")
class_name = data["theme"].get("class")

if selected_file is not None and class_name:
if selected_file and class_name:
module = importlib.import_module(selected_file[:-3])
obtained_class = getattr(module, class_name)
instance = obtained_class()
print(f"Theme {class_name} successfully loaded.")
return instance
elif selected_file == None and class_name:
elif class_name:
return class_name
else:
print("The theme is incorrect.")
Expand Down
5 changes: 2 additions & 3 deletions assets/version_checker.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,9 @@
import json
import requests

now_dir = os.getcwd()
sys.path.append(now_dir)
sys.path.append(os.getcwd())

config_file = os.path.join(now_dir, "assets", "config.json")
config_file = os.path.join(os.getcwd(), "assets", "config.json")


def load_local_version():
Expand Down
11 changes: 11 additions & 0 deletions assets/zluda/reinstall-torch.bat
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
set "principal=%cd%"
set "URL_EXTRA=https://huggingface.co/IAHispano/applio/resolve/main"
set "CONDA_ROOT_PREFIX=%UserProfile%\Miniconda3"
set "INSTALL_ENV_DIR=%principal%\env"
set "MINICONDA_DOWNLOAD_URL=https://repo.anaconda.com/miniconda/Miniconda3-py39_23.9.0-0-Windows-x86_64.exe"
set "CONDA_EXECUTABLE=%CONDA_ROOT_PREFIX%\Scripts\conda.exe"

call "%CONDA_ROOT_PREFIX%\condabin\conda.bat" activate "%INSTALL_ENV_DIR%"
rem pip uninstall torch torchvision torchaudio -y
rem pip install torch==2.1.1 torchvision==0.16.1 torchaudio==2.1.1 --index-url https://download.pytorch.org/whl/cu118
call "%CONDA_ROOT_PREFIX%\condabin\conda.bat" deactivate
Loading