Betydb changes?

[sckott]

hey @dlebauer anything we need to change here regarding https://zenodo.org/record/17128 ?

[dlebauer]

We are working on it in https://github.com/dlebauer/traits ... mostly to support development of this package, so if anything the current API should work better.

I'll review the recent changes and submit a pull request with our recent updates shortly.

[sckott]

@dlebauer great! sounds good

[dlebauer]

@sckott ... I've been reviewing some or the recent changes to the R functions by @phenolphtalein ... but they are broken ... so it may take a few more days. If you want to take a look, I am getting stuck on the way httr::GET creates a url.

... the code I've been working on with new (broken!) tests are in my dlebauer/query-fix branch https://github.com/dlebauer/traits/tree/dlebauer/query-fix

[sckott]

what part exactly? in general, GET takes as first param the base URL (e.g., http://stuff.com), then anything that comes after a ? goes into the query parameter (unless they are curl options or body values). commented on a commit

[dlebauer]

@sckott I added some tests, and thought they were passing - so I have two questions:

1. Is this repository hooked to Travis CI, or is there regular evaluation of tests?
2. I'd like to get these fixed. I am stuck (again) at

I want to search for Willow Vcmax data:

debugonce(betydb_http)
betydb_search("Salix Vcmax")

ultimately I get to

GET(url = "https://www.betydb.org/search.json", 
    args = list(search = "Salix+Vcmax"),
    authenticate(user = 'ropensci-traits', password = 'ropensci'))

Which does not restrict the query to either the species ('Salix') or the trait ('Vcmax'). Perhaps the url is not being formed correctly (I am looking for https://www.betydb.org/search.json?Salix+Vcmax)

[sckott]

hi @dlebauer

• Yes, Travis is hooked up. Builds at https://travis-ci.org/ropensci/traits/builds I just don't have the travis badge in the readme, I'll add that
• Not sure on that query, I'll take a look

[dlebauer]

@sckott

• Does Travis run the package tests (I've fixed them)?
• not sure why the query didn't work, but I think the answer is that GET function does some magic URL construction so I don't need to insert the + symbol. E.g.

GET(url = "https://www.betydb.org/search.json", 
    args = list(search = "Salix Vcmax"),
    authenticate(user = 'ropensci-traits', password = 'ropensci'))

works as expected ...

I'll send a pull request later with my updates

[dlebauer]

@sckott Let me know if I should post this on stackoverflow ... I am unfamilar with the httr package and a bit overwhelmed by httr_options().

I can start pasting strings together but also recognize that it is easier to let httr deal with special characters, etc., if it is able.

So, I want to paste together some include=[] strings and pass to the GET function?, for example, I want to construct the string

https://www.betydb.org/traits.json?genus=Acer&variables.name=leafN&include[]=specie&include[]=variable

I am assuming that I am passing the fields I want to parse in as a list, and they are in the format args = list(tablename.field = "value").

I want to get the singular version of the tablename and construct the url

This is what I have so far ... any assistance would be appreciated!

get_includes <- function(args) {
    ans <- list(`include[]=` = ifelse(any(grepl('species', names(args))), "specie", NULL),
                     `include[]=` = ifelse(any(grepl('variables', names(args))), 'variable'))
     return(ans)
}

args <- list(species.genus = "Acer", variables.name = "leafN")

includes <- get_includes(args)

GET(url = "https://www.betydb.org/search.json", 
    args = args,,
    ## where can I put the includes statement?
    authenticate(user = 'ropensci-traits', password = 'ropensci'))

[sckott]

@dlebauer hmmm, just trying URLs in a browser, I can get other endpoints to work, e.g https://www.betydb.org/traits.json , but /search.json doesn't seem to work in the browser

I don't think it's an httr problem

to pass parameters, use the query parameter in GET, so e.g,. GET(url, query = args) where args is a named list

[dlebauer]

when you say /search.json doesn't seem to work, I assume you mean https://www.betydb.org/search.json, which seems to work for me (if slow because the result is large). But https://www.betydb.org/search.json?genus=Acer should be faster.

the trouble I am having is in the named list, e.g. is this what should work?

args <- list(species.genus = 'Acer', `include[]=` = "specie", `include[]=` = "variable")

[sckott]

Ah, okay, maybe it just was taking a long time.

Yeah, I can't get those include[] variables to work. Again, that [] is very unusual. I'll keep digging to see what works

[dlebauer]

@gsrohde, @phenolphtalein is there an alternative to using the [] syntax in constructing the URL? Can it be omitted or somehow

[gsrohde]

@dlebauer The short answer is that you can program any API you want if one is willing to do the work. The advantage of the include[]= syntax is that that is the built-in way of representing an array of values connected with a single key. For example, if you had a form with a group of checkboxes or a multi-select box, the portion of the query string constructed from this would look like "key[]=value1&key[]=value2&key[]=value3 ", where "key" is the name of the variable associated with the checkbox group or the multi-select box and values 1, 2, and 3 are the values you selected.

That said, it's actually true that in general you don't need the square brackets if you only have one include in your query string, and this was working in BETYdb until I added some code to restore the output of included entities after the updgrade to Rails 3.2. In writing that code, I neglected to accommodate this use case, but it shouldn't be hard to do so if desired.

Looking back over the previous comments in this issue, I see that you are trying to use the "include[]=" syntax on the traits_and_yields_view. I don't think this will work at all without some additional work since we haven't told Rails to associate this table (view) with any other table.

[dlebauer]

Ok. It's not implemented for the search.json endpoint. that's okay. I'll
fiddle a bit more.
On Wed, May 13, 2015 at 1:55 PM Scott Rohde [email protected]
wrote:

@dlebauer https://github.com/dlebauer The short answer is that you can
program any API you want if one is willing to do the work. The advantage of
the include[]= syntax is that that is the built-in way of representing an
array of values connected with a single key. For example, if you had a form
with a group of checkboxes or a multi-select box, the portion of the query
string constructed from this would look like
"key[]=value1&key[]=value2&key[]=value3 ", where "key" is the name of the
variable associated with the checkbox group or the multi-select box and
values 1, 2, and 3 are the values you selected.

That said, it's actually true that in general you don't need the square
brackets if you only have one include in your query string, and this was
working in BETYdb until I added some code to restore the output of included
entities after the updgrade to Rails 3.2. In writing that code, I neglected
to accommodate this use case, but it shouldn't be hard to do so if desired.

Looking back over the previous comments in this issue, I see that you are
trying to use the "include[]=" syntax on the traits_and_yields_view. I
don't think this will work at all without some additional work since we
haven't told Rails to associate this table (view) with any other table.

—
Reply to this email directly or view it on GitHub
#25 (comment).

[sckott]

@gsrohde thanks for the feedback.

I see that you are trying to use the "include[]=" syntax on the traits_and_yields_view. I don't think this will work at all without some additional work

thanks, now that makes more sense

[sckott]

@dlebauer I'm removing trait related functions from taxize that have been moved here, and I'd like this package to be on CRAN soon so that users can still get to trait related fxns easily via CRAN

let's try to wrap up any betydb stuff soon for the v0.1 push to CRAN

[dlebauer]

When do you want to push to CRAN?

[sckott]

@dlebauer in the next week or two would be good

[dlebauer]

@sckott this issue can be closed ... I'll make new ones as needed

[sckott]

okay, thanks