Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How do I apply the puglin? #130

Open
seblemaguer opened this issue Jun 25, 2020 · 13 comments
Open

How do I apply the puglin? #130

seblemaguer opened this issue Jun 25, 2020 · 13 comments
Labels

Comments

@seblemaguer
Copy link
Member

How do I apply the puglin? I cloned marytts and run ./gradlew build that worked. I than cloned gradle-marytts-voicebuilding-plugin and run ./gradlew build again and that worked too, but adding the line to gradle.build (at gradle-marytts-voicebuilding-plugin/) does not work anymore...

Originally posted by @ingosiegert in #115 (comment)

@seblemaguer
Copy link
Member Author

Hello, what are you trying to do exactly?

@ingosiegert
Copy link

I want to train a voice, I already generated the project layout under gradle-marytts-voicebuilding-plugin. Using the ARCTIC data from CMU. So my steps were: clone marytts and clone gradle-marytts-voicebuilding-plugin then generate the project layout and run ./gradlew build. What did I miss?

@psibre
Copy link
Member

psibre commented Jun 26, 2020

To build a voice, you should need neither the marytts nor the voicebuilding plugin source code. You simply need a new Gradle project, which applies the voicebuilding plugin (retrieving it from the Gradle Plugins portal), which in turn configures all the default build logic for you -- including dependencies on marytts-runtime (resolved from JCenter).

Please have a look at some of our voice projects, e.g., voice-enst-camille or voice-cmu-slt.

@ingosiegert
Copy link

ingosiegert commented Jun 29, 2020

OK, I get It to run using the voice-cmu-slt voice-project. Now I wanted to redo it with local files. For testing, I used the voice-cmu-slt examples. I cloned the voice-cmu-slt git-project and copied the lab, text and wav files accordingly and generated a new "clean" build.gradle file:

plugins {
    id 'de.dfki.mary.voicebuilding-festvox' version '5.3.2'
    id 'de.dfki.mary.voicebuilding-legacy' version '5.3.2'
}

group 'de.dfki.mary'
version '5.3-SNAPSHOT'

voice {
    name = 'cmu-slt'
    gender = 'female'
    type = 'unit selection' 
    samplingRate = 16000
}

(I also deleted the three files that are excluded in the example).
But I got the following error:

:legacyHalfPhoneFeatureFileWriter
Loading database layout:
Reading config file /home/isiegert/Forschung/NDR/gradle-cmu/voice-cmu-slt/build/database.config
Loading basename list from wav files
Found 0 files in basename list
Loading features from file /home/isiegert/Forschung/NDR/gradle-cmu/voice-cmu-slt/build/mary/features.txt
Running HalfPhoneFeatureFileWriter
Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 0 >= 0
        at java.base/java.util.Vector.elementAt(Vector.java:496)
        at marytts.util.io.BasenameList.getName(BasenameList.java:364)
        at marytts.tools.voiceimport.PhoneFeatureFileWriter.compute(PhoneFeatureFileWriter.java:103)
        at marytts.tools.voiceimport.DatabaseImportMain.main(DatabaseImportMain.java:428)
:legacyHalfPhoneFeatureFileWriter FAILED

What is the problem?

@ingosiegert
Copy link

ingosiegert commented Jun 30, 2020

I maybe found the solution to this error. By generating a basename.lst using the following configuration:

plugins {
    id 'de.dfki.mary.voicebuilding-festvox' version '5.3.2'
    id 'de.dfki.mary.voicebuilding-legacy' version '5.3.2'
}

group 'de.dfki.mary'
version '5.3-SNAPSHOT'

voice {
    name = 'cmu-slt'
    gender = 'female'
    type = 'unit selection' 
    samplingRate = 16000
}

task generateBasenamesList {
    def destFile = file("$buildDir/basenames.lst")
    outputs.files destFile
    doLast {
        destFile.text = fileTree("$buildDir/wav").include('*.wav').collect {
            it.name - '.wav'
        }.join('\n')
    }
}

legacyInit.dependsOn generateBasenamesList

and running ./gradlew legacyInit.
But now I get another error:

        at marytts.util.data.ESTTrackReader.<init>(ESTTrackReader.java:90)
        at marytts.tools.voiceimport.PhoneUnitfileWriter.compute(PhoneUnitfileWriter.java:107)
        at marytts.tools.voiceimport.DatabaseImportMain.main(DatabaseImportMain.java:428)
:legacyHalfPhoneUnitfileWriter FAILED

Why are these files not generated, needed at this point? When copying the files from the git-cloned version, it runs through fine and generates the voice.

@seblemaguer
Copy link
Member Author

Sorry but there is not enough information for us to debug here. Could you paste the full stacktrace?

@ingosiegert
Copy link

ingosiegert commented Jun 30, 2020

Sure:

Exception in thread "main" java.lang.Error: EST track file [/home/isiegert/Forschung/NDR/gradle-cmu/voice-cmu-slt/build/pm/arctic_a0302.pm] was not found.
        at marytts.util.data.ESTTrackReader.<init>(ESTTrackReader.java:90)
        at marytts.tools.voiceimport.PhoneUnitfileWriter.compute(PhoneUnitfileWriter.java:107)
        at marytts.tools.voiceimport.DatabaseImportMain.main(DatabaseImportMain.java:428)
:legacyHalfPhoneUnitfileWriter FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':legacyHalfPhoneUnitfileWriter'.
> Process 'command '/usr/lib/jvm/java-10-openjdk/bin/java'' finished with non-zero exit value 1

* Try:
Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Exception is:
org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':legacyHalfPhoneUnitfileWriter'.
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:103)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:73)
        at org.gradle.api.internal.tasks.execution.OutputDirectoryCreatingTaskExecuter.execute(OutputDirectoryCreatingTaskExecuter.java:51)
        at org.gradle.api.internal.tasks.execution.SkipUpToDateTaskExecuter.execute(SkipUpToDateTaskExecuter.java:59)
        at org.gradle.api.internal.tasks.execution.ResolveTaskOutputCachingStateExecuter.execute(ResolveTaskOutputCachingStateExecuter.java:54)
        at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:59)
        at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:101)
        at org.gradle.api.internal.tasks.execution.FinalizeInputFilePropertiesTaskExecuter.execute(FinalizeInputFilePropertiesTaskExecuter.java:44)
        at org.gradle.api.internal.tasks.execution.CleanupStaleOutputsExecuter.execute(CleanupStaleOutputsExecuter.java:88)
        at org.gradle.api.internal.tasks.execution.ResolveTaskArtifactStateTaskExecuter.execute(ResolveTaskArtifactStateTaskExecuter.java:62)
        at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:52)
        at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:54)
        at org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter.execute(ExecuteAtMostOnceTaskExecuter.java:43)
        at org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:34)
        at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter$EventFiringTaskWorker$1.run(DefaultTaskGraphExecuter.java:248)
        at org.gradle.internal.progress.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:336)
        at org.gradle.internal.progress.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:328)
        at org.gradle.internal.progress.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:199)
        at org.gradle.internal.progress.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:110)
        at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter$EventFiringTaskWorker.execute(DefaultTaskGraphExecuter.java:241)
        at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter$EventFiringTaskWorker.execute(DefaultTaskGraphExecuter.java:230)
        at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$TaskExecutorWorker.processTask(DefaultTaskPlanExecutor.java:123)
        at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$TaskExecutorWorker.access$200(DefaultTaskPlanExecutor.java:79)
        at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$TaskExecutorWorker$1.execute(DefaultTaskPlanExecutor.java:104)
        at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$TaskExecutorWorker$1.execute(DefaultTaskPlanExecutor.java:98)
        at org.gradle.execution.taskgraph.DefaultTaskExecutionPlan.execute(DefaultTaskExecutionPlan.java:623)
        at org.gradle.execution.taskgraph.DefaultTaskExecutionPlan.executeWithTask(DefaultTaskExecutionPlan.java:578)
        at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$TaskExecutorWorker.run(DefaultTaskPlanExecutor.java:98)
        at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:63)
        at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:46)
        at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:55)
Caused by: org.gradle.process.internal.ExecException: Process 'command '/usr/lib/jvm/java-10-openjdk/bin/java'' finished with non-zero exit value 1
        at org.gradle.process.internal.DefaultExecHandle$ExecResultImpl.assertNormalExitValue(DefaultExecHandle.java:389)
        at org.gradle.process.internal.DefaultJavaExecAction.execute(DefaultJavaExecAction.java:36)
        at org.gradle.api.internal.file.DefaultFileOperations.javaexec(DefaultFileOperations.java:183)
        at org.gradle.api.internal.project.DefaultProject.javaexec(DefaultProject.java:1077)
        at org.gradle.api.internal.project.DefaultProject.javaexec(DefaultProject.java:1072)
        at org.gradle.api.Project$javaexec$7.call(Unknown Source)
        at de.dfki.mary.voicebuilding.tasks.LegacyVoiceImportTask.run(LegacyVoiceImportTask.groovy:46)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:73)
        at org.gradle.api.internal.project.taskfactory.StandardTaskAction.doExecute(StandardTaskAction.java:46)
        at org.gradle.api.internal.project.taskfactory.StandardTaskAction.execute(StandardTaskAction.java:39)
        at org.gradle.api.internal.project.taskfactory.StandardTaskAction.execute(StandardTaskAction.java:26)
        at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:788)
        at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:755)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$1.run(ExecuteActionsTaskExecuter.java:124)
        at org.gradle.internal.progress.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:336)
        at org.gradle.internal.progress.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:328)
        at org.gradle.internal.progress.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:199)
        at org.gradle.internal.progress.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:110)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeAction(ExecuteActionsTaskExecuter.java:113)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:95)
        ... 30 more


* Get more help at https://help.gradle.org

BUILD FAILED in 1s
19 actionable tasks: 1 executed, 18 up-to-date

@seblemaguer
Copy link
Member Author

It says that this file "/home/isiegert/Forschung/NDR/gradle-cmu/voice-cmu-slt/build/pm/arctic_a0302.pm" doesn't exist. do you have anything in the pm directory? Anyhow, double check if you didn't have a warning or anything before.

@ingosiegert
Copy link

Yes, I think that is the problem. The pm directory is indeed empty. When should it be filled, by which process?

@ingosiegert
Copy link

Find attached the full stacktrace:

trace.txt

@ingosiegert
Copy link

ingosiegert commented Jun 30, 2020

I had a deeper look at the stacktrace output and found the following message:
:legacyHalfPhoneUnitfileWriter Loading database layout: Reading config file /home/isiegert/Forschung/NDR/gradle-cmu/voice-cmu-slt/build/database.config Loading basename list from file /home/isiegert/Forschung/NDR/gradle-cmu/voice-cmu-slt/build/basenames.lst Found 1129 files in basename list Loading features from file /home/isiegert/Forschung/NDR/gradle-cmu/voice-cmu-slt/build/mary/features.txt Running HalfPhoneUnitfileWriter Unitfile writer started. Verifying that unit feature and label files are perfectly aligned... Verifying feature-label alignment for 1129 utterances. arctic_a0302 No feature file /home/isiegert/Forschung/NDR/gradle-cmu/voice-cmu-slt/build/halfphonefeatures/arctic_a0302.hpfeats

Could that be an possible issue?

Although the files are there:
ls -la /home/isiegert/Forschung/NDR/gradle-cmu/voice-cmu-slt/build/halfphonefeatures/arctic_a0302.hpfeats -rw-r--r-- 1 isiegert isiegert 53376 30. Jun 14:48 /home/isiegert/Forschung/NDR/gradle-cmu/voice-cmu-slt/build/halfphonefeatures/arctic_a0302.hpfeats

@psibre
Copy link
Member

psibre commented Jul 1, 2020

Thanks for investigating! Indeed it looks like there are certain corner cases where the intermediate files generated during the voicebuilding steps do not align properly. We actually avoided this issue (and never got around to investigating and fixing the root issue) by simply excluding certain utterances from the basenames.lst that controls which utterances are processed. E.g., in voice-cmu-slt, we added this hack:

task generateBasenamesList {
    def destFile = file("$buildDir/basenames.lst")
    outputs.files destFile
    dependsOn wav
    doLast {
        destFile.text = fileTree("$buildDir/wav").include('*.wav').collect {
            it.name - '.wav'
        }.minus([
                'arctic_a0438',
                'arctic_a0439',
                'arctic_b0391'
        ]).join('\n')
    }
}

It appears that upgrading that voice to the latest version of the voicebuilding plugin exacerbates the original problem, and hacking the basenames.lst further until all problematic utterances are excluded is infeasible and pointless. We'll have to investigate this further when we have the time...

@ingosiegert
Copy link

@psibre Yes you are right. during all my tests, I was aware of this hack and totally forget about it, as I thought I directly used a reduced set of data, which was not the case. So now it is successfully building the voice also with local files.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants