Skip to content

example doesn't work on Debian with installed freeling v3.1 #25

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
filterfish opened this issue Mar 26, 2015 · 2 comments
Open

example doesn't work on Debian with installed freeling v3.1 #25

filterfish opened this issue Mar 26, 2015 · 2 comments

Comments

@filterfish
Copy link

I tried the example from the README but it returns an empty array. FreeLing is installed from the dpkg downloaded from the FreeLing site. MeCab isn't installed.

dpkg -l | grep freeling
ii  freeling     3.1    amd64        Library of natural language analysis facilities.

I piped the following through pry:

require 've'
provider = Ve::Manager.provider_for(:en, :words)
parse = provider.parse('I like melons.')
parse.send(:words)

I get:

[1] pry(main)> require 've'
/home/rgh/.gem/ruby/2.2.0/gems/ve-0.0.3/lib/providers/japanese_transliterators.rb:100: warning: duplicated key at line 101 ignored: "gwi"
/home/rgh/.gem/ruby/2.2.0/gems/ve-0.0.3/lib/providers/japanese_transliterators.rb:100: warning: duplicated key at line 101 ignored: "gwu"
/home/rgh/.gem/ruby/2.2.0/gems/ve-0.0.3/lib/providers/japanese_transliterators.rb:100: warning: duplicated key at line 101 ignored: "gwe"
/home/rgh/.gem/ruby/2.2.0/gems/ve-0.0.3/lib/providers/japanese_transliterators.rb:100: warning: duplicated key at line 101 ignored: "gwo"
=> true
[2] pry(main)> provider = Ve::Manager.provider_for(:en, :words)
=> #<Ve::Provider::FreelingEn:0x007fdb841021d0
 @config={:app=>"/usr/bin/analyzer", :path=>"", :flags=>"-f /usr/share/FreeLing/config/en.cfg --flush --nonumb --nodate"},
 @stderr=#<IO:fd 12>,
 @stdin=#<IO:fd 9>,
 @stdout=#<IO:fd 10>>
[3] pry(main)> parse = provider.parse('I like melons.')
=> #<Ve::Parse::FreelingEn:0x007fdb84239080 @text="I like melons.", @tokens=[]>
[4] pry(main)> parse.send(:words)
=> []
[5] pry(main)> 

I initially thought it was -f /usr/share/FreeLing/config/en.cfg flag as this path doesn't exist. The path is /usr/share/freeling/config/en.cfg. So I'm symlinked /usr/share/FreeLing to /usr/share/freeling but it made no difference.

If I run it on the command line it works analyze seems to return something sensible (I've never used FreeLing before):

echo "I like melons." | analyze --locale=en_AU.UTF-8 -f /usr/share/FreeLing/config/en.cfg --flush --nonumb --nodate
I i PRP 1
like like IN 0.867653
melons melon NNS 1
. . Fp 1

I wondered if it might be the locale option (I don't have en_US.UTF-8 installed) and analyze fails without it but after adding locale it still returned an empty array.

@Kimtaro
Copy link
Owner

Kimtaro commented Mar 27, 2015

Hmm, it might be that something has changed in FreeLing 3.1. I think it was on version 2.9 when I wrote this code. I'll see if I can reproduce locally with 3.1 and fix.

@Kimtaro
Copy link
Owner

Kimtaro commented Mar 28, 2015

I have tried with Freeling 3.1 and it's working for me.

It's odd that your Ve is trying to use /usr/share/FreeLing when it should use /usr/share/freeling though.

Try forcing the provider class to use the correct path by running this right after require 've': Ve::Manager.set_default_config_for(Ve::Provider::FreelingEn, {:flags => "-f /usr/local/share/freeling/config/en.cfg --flush --nonumb --nodate"})

Then try

provider = Ve::Manager.provider_for(:en, :words)
parse = provider.parse('I like melons.')

Here's the output I get:

$ bundle exec irb -Ilib
irb(main):001:0> require 've'
=> true
irb(main):002:0> Ve::Manager.set_default_config_for(Ve::Provider::FreelingEn, {:flags => "-f /usr/local/share/freeling/config/en.cfg --flush --nonumb --nodate"})
=> {:flags=>"-f /usr/local/share/freeling/config/en.cfg --flush --nonumb --nodate"}
irb(main):003:0> provider = Ve::Manager.provider_for(:en, :words)
=> #<Ve::Provider::FreelingEn:0x007f0fad04c6d0 @config={:app=>"/usr/local/bin/analyzer", :path=>"", :flags=>"-f /usr/local/share/freeling/config/en.cfg --flush --nonumb --nodate", :freeling_dir_name=>"freeling"}, @stdin=#<IO:fd 10>, @stdout=#<IO:fd 11>, @stderr=#<IO:fd 13>>
irb(main):004:0> parse = provider.parse('I like melons.')
I i PRP 1
like like IN 0.867653
melons melon NNS 1
. . Fp 1

VeEnd veend NP 1
=> #<Ve::Parse::FreelingEn:0x007f0fad03e918 @tokens=[{:raw=>"I i PRP 1", :type=>:parsed, :literal=>"I", :lemma=>"i", :pos=>"PRP", :accuracy=>"1", :characters=>0..0}, {:type=>:unparsed, :literal=>" ", :characters=>1..1}, {:raw=>"like like IN 0.867653", :type=>:parsed, :literal=>"like", :lemma=>"like", :pos=>"IN", :accuracy=>"0.867653", :characters=>2..5}, {:type=>:unparsed, :literal=>" ", :characters=>6..6}, {:raw=>"melons melon NNS 1", :type=>:parsed, :literal=>"melons", :lemma=>"melon", :pos=>"NNS", :accuracy=>"1", :characters=>7..12}, {:raw=>". . Fp 1", :type=>:parsed, :literal=>".", :lemma=>".", :pos=>"Fp", :accuracy=>"1", :characters=>13..13}, {:raw=>"", :type=>:sentence_split, :literal=>""}], @text="I like melons.">
irb(main):005:0>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants