Read camtrap-dp data with camtrapR

[damianooldoni]

Camera Trap Data Package (Camtrap DP) has been greatly developed in the last months. And it will become a TDWG standard (see https://gitlab.com/oscf/camtrap-dp/-/issues/85) soon as it is mature enough. This means the use of this data format will greatly increase in the near future. Agouti uses already this format to export its data.

What do you think about a read function in camtrapR for camtrap-dp formatted data? Is it already part of your dev branch? If not, I and other INBO colleagues (@jimcasaer and its colleagues of the Fauna and Invasive Species team) we are quit interested to work on this. Thanks.

[damianooldoni]

Update: camtrap-dp is already a TDWG standard. See https://tdwg.github.io/camtrap-dp/

[jniedballa]

Hi,
is the definition of the standard finished? And do you expect the standard to evolve further in the future, requiring ongoing updates?
So far I didn't do any work in this direction within camtrapR, but would of course be open to discuss how to approach this. From what I see there are some key differences:

• camtrapR combines the content of multimedia.csv and observations.csv in one record table
• camtrapR doesn't require a metadata table
• the format of the deployment table differs, in particular the use of the Problem columns in the camera trap tables of camtrapR is not compatible with the standard, but essential to many function within camtrapR.

Right now I am hesitant to change core functionalities of camtrapR that many people got used to and have code for. So I'd suggest we find a way to convert the data automatically between the two.

There might be more issues when digging into the details. But I'm sure we can write an export function that takes care of these things.
Anyways, I'm happy to discuss about this.

Cheers,
Jürgen

[jimcasaer]

Hi Jürgen,

I fully agree that you should not change the core functionalities of camtrapR given the number of users of the package at the moment. I think the way forward is that we look together how we can import/translate the data from deployments / observations of the new export format to the existing objects in camtrapR and add new functionalities strating from existing objects in camtrapR where possible.

I will share a google-doc with you next week that is a working document so you can see the total set of data-exploration and data-visualisation tools we are aiming for.

have a nice weekend,
Jim

[damianooldoni]

Indeed, @jniedballa: the point is to allow users to read data from camtrap-dp standard in camtrapR, so they can use camtrapR package to analyze them. The way data are stored in camtrapR should not change unless you see the need for it in the future.

About standard stability, I think it is quite stable up to now, However, it is far from being a mature standard. @peterdesmet: as you are actively involved in the standard development/maintenance, can you confirm this?

Thanks.

[peterdesmet]

Indeed, even though camtrap-dp is now hosted under Biodiversity Information Standards (TDWG), it is not a standard yet. Pending one issue with the metadata, it is a 1.0 release. @kbubnicki and I have worked hard to get it there and it is now the only export format for https://agouti.eu, meaning many users will get their hands on it. That is a good thing, because it brings the format from theory into practice, and test how well it meets different use cases. It also means fields might get added in the future.

What would be really helpful to users though is that some of the complexity/development is hidden to them and they can just read the format with camtrapR and work with the camtrapR objects they are familiar with (as mentioned by @damianooldoni and @jimcasaer above). That would also avoid users/developers writing their own functions to work with the format.

[kbubnicki]

Hi all!
I also think that the implementation of some user-friendly import/export interface (2 functions?) for reading/writing camtrap-dp data packages directly to/from camtrapR would be wonderful, especially thinking about how popular and useful is camtrapR already! Many of our TRAPPER users are very interested in this integration. It could also help with building semi-automatic camera trapping data analysis pipelines based on e.g. Agouti and TRAPPER as data providers (via API) and camtrapR as a direct interface to the R analytical environment.

[jniedballa]

Hi,
sure, sound good to me. Generally an import / export functionality sounds like a reasonable thing to implement.
I'll wait for the GDoc Jim mentioned, then we can discuss in more detail what is required.