Skip to content

Support for both OpenFST 1.7.3 and 1.8.2 #4927

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Sep 8, 2024
Merged

Support for both OpenFST 1.7.3 and 1.8.2 #4927

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
3f170d4
Support openfst-1.7.6
georgthegreat Feb 23, 2023
e460d8a
Support openfst-1.8.0
georgthegreat Feb 23, 2023
5ccce55
Support openfst-1.8.1
georgthegreat Feb 23, 2023
9a213be
Support openfst-1.8.2
georgthegreat Feb 23, 2023
57efb6b
add support for later openfst versions
jtrmal Jul 25, 2024
a7fcc2d
fix the non-compiling tests, thx to @csukuangfj
jtrmal Jul 25, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion src/base/kaldi-error-test.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ int main() {
kaldi::UnitTestError();
KALDI_ASSERT(0); // should not happen.
exit(1);
} catch (kaldi::KaldiFatalError &e) {
} catch (const kaldi::KaldiFatalError &e) {
std::cout << "The error we generated was: '" << e.KaldiMessage() << "'\n";
}
}
35 changes: 14 additions & 21 deletions src/base/kaldi-types.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,9 +39,21 @@ typedef float BaseFloat;
// we find in the future lacks stdint.h
#include <stdint.h>

// for discussion on what to do if you need compile kaldi
// without OpenFST, see the bottom of this this file
#if OPENFST_VER >= 10800
typedef int8_t int8;
typedef int16_t int16;
typedef int32_t int32;
typedef int64_t int64;

typedef uint8_t uint8;
typedef uint16_t uint16;
typedef uint32_t uint32;
typedef uint64_t uint64;
typedef float float32;
typedef double double64;
#else
#include <fst/types.h>
#endif

namespace kaldi {
using ::int16;
Expand All @@ -53,23 +65,4 @@ namespace kaldi {
typedef float float32;
typedef double double64;
} // end namespace kaldi

// In a theoretical case you decide compile Kaldi without the OpenFST
// comment the previous namespace statement and uncomment the following
/*
namespace kaldi {
typedef int8_t int8;
typedef int16_t int16;
typedef int32_t int32;
typedef int64_t int64;

typedef uint8_t uint8;
typedef uint16_t uint16;
typedef uint32_t uint32;
typedef uint64_t uint64;
typedef float float32;
typedef double double64;
} // end namespace kaldi
*/

#endif // KALDI_BASE_KALDI_TYPES_H_
3 changes: 2 additions & 1 deletion src/bin/phones-to-prons.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -172,7 +172,8 @@ int main(int argc, char *argv[]) {
if (g_kaldi_verbose_level >= 2) {
KALDI_LOG << "phn2word FST is below:";
fst::FstPrinter<StdArc> fstprinter(phn2word, NULL, NULL, NULL, false, true, "\t");
fstprinter.Print(&std::cerr, "standard error");
printer_print(std::cerr, fstprinter, "standard error");
//fstprinter.Print(&std::cerr, "standard error");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove this?

KALDI_LOG << "phone sequence is: ";
for (size_t i = 0; i < phones.size(); i++)
std::cerr << phones[i] << ' ';
Expand Down
10 changes: 4 additions & 6 deletions src/chain/chain-supervision.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -571,9 +571,8 @@ void Supervision::Write(std::ostream &os, bool binary) const {
// Write using StdAcceptorCompactFst, making use of the fact that it's an
// acceptor.
fst::FstWriteOptions write_options("<unknown>");
fst::StdCompactAcceptorFst::WriteFst(
fst, fst::AcceptorCompactor<fst::StdArc>(), os,
write_options);
fst::StdCompactAcceptorFst cfst(fst);
cfst.Write(os, write_options);
Comment on lines +574 to +575
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this pre-1.8 compatible? Should we go flat out 1.8+? We've had the version of OpenFST fixed pretty much forever.

}
} else {
KALDI_ASSERT(e2e_fsts.size() == num_sequences);
Expand All @@ -586,9 +585,8 @@ void Supervision::Write(std::ostream &os, bool binary) const {
// Write using StdAcceptorCompactFst, making use of the fact that it's an
// acceptor.
fst::FstWriteOptions write_options("<unknown>");
fst::StdCompactAcceptorFst::WriteFst(
e2e_fsts[i], fst::AcceptorCompactor<fst::StdArc>(), os,
write_options);
fst::StdCompactAcceptorFst cfst(e2e_fsts[i]);
cfst.Write(os, write_options);
}
}
WriteToken(os, binary, "</Fsts>");
Expand Down
10 changes: 9 additions & 1 deletion src/configure
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@

# This should be incremented after any significant change to the configure
# script, i.e. any change affecting kaldi.mk or the build system as a whole.
CONFIGURE_VERSION=14
CONFIGURE_VERSION=15

# We support bash version 3.2 (Macs still ship with this version as of 2019)
# and above.
Expand Down Expand Up @@ -1024,6 +1024,14 @@ OPENFST_VER_NUM=$(echo $OPENFST_VER | sed 's/\./ /g' | xargs printf "%d%02d%02d"
if [ $OPENFST_VER_NUM -lt 10600 ]; then
failure "OpenFst-$OPENFST_VER is not supported. You need OpenFst >= 1.6.0.)"
fi

if [ $OPENFST_VER_NUM -lt 10800 ]; then
echo "CXXLANGVERSION = c++14"
else
echo "CXXLANGVERSION = c++17"
fi >> kaldi.mk
Comment on lines +1028 to +1032
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you think of a case when we would run into a pre-C++14 compiler? Maybe it's time to go C++17 unconditionally?


echo "OPENFSTVER = $OPENFST_VER_NUM" >> kaldi.mk
echo "OPENFSTINC = $FSTROOT/include" >> kaldi.mk
if $static_fst ; then
OPENFSTLIBS="$FSTROOT/lib/libfst.a"
Expand Down
6 changes: 4 additions & 2 deletions src/fstext/context-fst-test.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@
#include "util/kaldi-io.h"
#include "base/kaldi-math.h"

#include "fstext/openfst_compat.h"

namespace fst
{
using std::vector;
Expand Down Expand Up @@ -196,7 +198,7 @@ static void TestContextFst(bool verbose, bool use_matcher) {
std::cout << "Sequence FST is:\n";
{ // Try to print the fst.
FstPrinter<Arc> fstprinter(*f, NULL, NULL, NULL, false, true, "\t");
fstprinter.Print(&std::cout, "standard output");
printer_print(std::cout, fstprinter, "standard output");
}
}

Expand Down Expand Up @@ -224,7 +226,7 @@ static void TestContextFst(bool verbose, bool use_matcher) {
std::cout << "Composed FST is:\n";
{ // Try to print the fst.
FstPrinter<Arc> fstprinter(fst_composed, NULL, NULL, NULL, false, true, "\t");
fstprinter.Print(&std::cout, "standard output");
printer_print(std::cout, fstprinter, "standard output");
}
}

Expand Down
14 changes: 8 additions & 6 deletions src/fstext/determinize-lattice-test.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@
#include "fstext/fst-test-utils.h"
#include "base/kaldi-math.h"

#include "fstext/openfst_compat.h"

Comment on lines 22 to +26
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sort #includes alphabetically. Or, if openfst_compat.h requires being included the last, then add a comment to that effect.

namespace fst {
using std::vector;
using std::cout;
Expand Down Expand Up @@ -94,7 +96,7 @@ template<class Arc> void TestDeterminizeLattice() {
std::cout << "FST before lattice-determinizing is:\n";
{
FstPrinter<Arc> fstprinter(*fst, NULL, NULL, NULL, false, true, "\t");
fstprinter.Print(&std::cout, "standard output");
printer_print(std::cout, fstprinter, "standard output");
}
VectorFst<Arc> det_fst;
try {
Expand All @@ -106,7 +108,7 @@ template<class Arc> void TestDeterminizeLattice() {
std::cout << "FST after lattice-determinizing is:\n";
{
FstPrinter<Arc> fstprinter(det_fst, NULL, NULL, NULL, false, true, "\t");
fstprinter.Print(&std::cout, "standard output");
printer_print(std::cout, fstprinter, "standard output");
}
assert(det_fst.Properties(kIDeterministic, true) & kIDeterministic);
// OK, now determinize it a different way and check equivalence.
Expand All @@ -117,7 +119,7 @@ template<class Arc> void TestDeterminizeLattice() {
std::cout << "Compact FST is:\n";
{
FstPrinter<CompactArc> fstprinter(compact_fst, NULL, NULL, NULL, false, true, "\t");
fstprinter.Print(&std::cout, "standard output");
printer_print(std::cout, fstprinter, "standard output");
}
if (kaldi::Rand() % 2 == 1)
ConvertLattice<Weight, Int>(det_fst, &compact_det_fst, false);
Expand All @@ -128,7 +130,7 @@ template<class Arc> void TestDeterminizeLattice() {
std::cout << "Compact version of determinized FST is:\n";
{
FstPrinter<CompactArc> fstprinter(compact_det_fst, NULL, NULL, NULL, false, true, "\t");
fstprinter.Print(&std::cout, "standard output");
printer_print(std::cout, fstprinter, "standard output");
}

assert(RandEquivalent(compact_det_fst, compact_fst, 5/*paths*/, 0.01/*delta*/, kaldi::Rand()/*seed*/, 100/*path length, max*/));
Expand All @@ -149,14 +151,14 @@ template<class Arc> void TestDeterminizeLattice2() {
std::cout << "FST before lattice-determinizing is:\n";
{
FstPrinter<Arc> fstprinter(*fst, NULL, NULL, NULL, false, true, "\t");
fstprinter.Print(&std::cout, "standard output");
printer_print(std::cout, fstprinter, "standard output");
}
VectorFst<Arc> ofst;
DeterminizeLattice<TropicalWeight, int32>(*fst, &ofst);
std::cout << "FST after lattice-determinizing is:\n";
{
FstPrinter<Arc> fstprinter(ofst, NULL, NULL, NULL, false, true, "\t");
fstprinter.Print(&std::cout, "standard output");
printer_print(std::cout, fstprinter, "standard output");
}
delete fst;
}
Expand Down
Loading
Loading