joint models for metric data

[smar-git]

Hi,

I am working on a problem were we have two types of observation on the street network, they are both point observations but with different observation error.

I though that would be easy to implement in the inlabru framework using two like() functions and two different dataset, as one usually does for joint models. The problem is that, if one wants to use the MetricGraph library, the data are stored together with the spde model so it is not possible to use the “usual” approach, or at least that is my impression.

Is this correct? Or how would you implement these types of model?

I also have another question, in the paper, you fit a model with alpha =2 but if I try this I get the message that the model is not implemented

spde_model <- graph_spde(graph, alpha =2)

Error in graph_spde(graph, alpha = 2) : Only alpha=1 implemented.

Is this correct?

Thanks for help

Bests

Sara

[AlexandreBSimas]

Hi Sara,

I suggest you use our process_data() method to build the data you need inside the metricgraph format.

I will add more examples in the vignette now, so you can see how it works (I will let you know here when I finish adding it). But, it is basically identical to add_observations(), but instead of adding the data to the graph, it returns the data frame in the graph's internal storage format, so you can pass it to inlabru.

You can see an example in https://davidbolin.github.io/MetricGraph/articles/MetricGraph.html (you can search for graph$process_data( ).

With respect to the second question. It is true, unfortunately, for the moment we only have alpha=1 implemented in INLA. We will be implementing soon the alpha=2 case, and also rational approximations (without finite element) in INLA. I will send you a message here when we finish implementing it.

If you want to fit this model (in a non-bayesian way), you can use the rspde_lme() function. You can see examples here: https://davidbolin.github.io/MetricGraph/articles/comparison.html

[smar-git]

hi
this is the way i am fitting the model now

graph$add_observations(data = df_graph, normalized = T)
cmp <- ~ -1 + Intercept(1) + cov(covariate, model = "linear") + field(loc,
model = spde_model)
data_spde <- graph_data_spde(spde_model, loc_name = "loc")
lik1 = like(formula = y ~.,
data=data_spde[["data"]])
spde_bru_fit <- bru(cmp, lik1)

if i understand what you are suggesting here is that i should create a new data object

data1 = graph$process_data(data = df_graph)

and pass this to the like() function instaed of the data_spde[["data"]], is this correct?

what is missing is the information about what "loc" is so i get an error. Should i manually add this? or should i change the name in the components for the spde model?

[AlexandreBSimas]

Yes, exactly. You can either add the location manually, or use the function (that essentially only does it): graph_bru_process_data()

You can see the content of the function here:

graph_bru_process_data <- function(data, edge_number = "edge_number",
                                        distance_on_edge = "distance_on_edge",
                                        loc = "loc"){

                                        data[[loc]] <- cbind(data[[edge_number]], data[[distance_on_edge]])
                                        data[[edge_number]] <- NULL
                                        data[[distance_on_edge]] <- NULL
                                        return(data)
                                        }

[smar-git]

Hi Our problem is that we are trying to fit a model where the data are not observations on a point ( as in your example) but averages over lines, when david was here in Trondheim he says that these type of observations cannot be stored in the graph (at least not the way the library is now) so what we do is to build our own A matrix to put in control.predictor=list(A=inla.stack.A(stk_dat))) that would compute the averages over the lines of interest. This works in the inla framework but not in the inlabru as inlabru overwrites the A matrix that one provide. Do you see a better way of doing that? Sara From: Alexandre B. Simas ***@***.***> Date: Tuesday, 4 June 2024 at 09:22 To: davidbolin/MetricGraph ***@***.***> Cc: Sara Martino ***@***.***>, Author ***@***.***> Subject: Re: [davidbolin/MetricGraph] joint models for metric data (Discussion #5) Hi Sara, I suggest you use our process_data() method to build the data you need inside the metricgraph format. I will add more examples in the vignette now, so you can see how it works (I will let you know here when I finish adding it). But, it is basically identical to add_observations(), but instead of adding the data to the graph, it returns the data frame in the graph's internal storage format, so you can pass it to inlabru. You can see an example in https://davidbolin.github.io/MetricGraph/articles/MetricGraph.html (you can search for graph$process_data( ). With respect to the second question. It is true, unfortunately, for the moment we only have alpha=1 implemented in INLA. We will be implementing soon the alpha=2 case, and also rational approximations (without finite element) in INLA. I will send you a message here when we finish implementing it. If you want to fit this model (in a non-bayesian way), you can use the rspde_lme() function. You can see examples here: https://davidbolin.github.io/MetricGraph/articles/comparison.html — Reply to this email directly, view it on GitHub<#5 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ANIDL5ODR4PYWDZQRLOYZ3DZFVTJDAVCNFSM6AAAAABIWFKQNSVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4TMNJZHE2DS>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[AlexandreBSimas]

When you say "averages over lines" are these values constant over edges? Or can you have different values on a same edge?

[AlexandreBSimas]

How is bru overwriting the matrix? Which model are you passing there, the same spde_model? If you create a new cgeneric class you can create your own ibm_jacobian in inlabru, and it would not overwrite it.

[AlexandreBSimas]

Basically, if you are using, for instance, graph_spde to create your model to be fitted, it will have class inla_metric_graph_spde.
When you pass this model to inlabru, it will automatically assemble the matrix using our function ibm_jacobian.bru_mapper_inla_metric_graph_spde.

So my suggestion is for you to create a new class (even if you use our same graph_spde, you can either create graph_spde2 or something, or directly overwrite the class as:
class(spde_model) <- c("new_name_class", "inla.cgeneric")

Then, you create new mappers (you can use the same mappers, and just replace the ibm_jacobian, which gives the A matrix:

bru_get_mapper.new_name_class <- function(model, ...){
  mapper <- list(model = model)
  inlabru::bru_mapper_define(mapper, new_class = "bru_mapper_new_name_class")
}

#' @param mapper A `bru_mapper.new_name_class` object
#' @rdname bru_mapper.new_name_class
ibm_n.bru_mapper_new_name_class <- function(mapper, ...) {
  model <- mapper[["model"]]
  n_groups <- length(unique(model$graph_spde$.__enclos_env__$private$data[[".group"]]))
  return(model$f$n)
}
#' @rdname bru_mapper.new_name_class
ibm_values.bru_mapper_new_name_class <- function(mapper, ...) {
  seq_len(inlabru::ibm_n(mapper))
}
#' @param input The values for which to produce a mapping matrix
#' @rdname bru_mapper.inla_metric_graph_spde
ibm_jacobian.bru_mapper_new_name_class <- function(mapper, input, ...) {
        # Your code here.
}

(you can see the codes here: https://github.com/davidbolin/MetricGraph/blob/devel/R/graph_spde.R)

[AlexandreBSimas]

Just observe that you need to register these as S3 methods. So, it is better to do this inside a package.

[smar-git]

Hi, what do you mean with "is better to do this inside a package." ?

[AlexandreBSimas]

Hi Sara,

Sorry, I did not see this message before.

What I mean is that, you want to use our latent field (the whittle-matern field with alpha=1 or alpha=2), but use your A matrix in inlabru, you need to create that ibm_jacobian function. But this needs to be a registered S3 method.

So, the easiest way to do that, is for you to create a new R package, and add those inlabru methods as S3 methods from your package. Then, when you run inlabru, it will recognize the methods and use your A matrix defined in your ibm_jacobian.

[smar-git]

They can be different…but you only observe the average. The case study we are working on is bus speed. You only observe the time the leave the butstop and the time they arrive at the next one. Since you know the distance you can then compute the average speed…but no the speed at one point. From: Alexandre B. Simas ***@***.***> Date: Tuesday, 4 June 2024 at 09:39 To: davidbolin/MetricGraph ***@***.***> Cc: Sara Martino ***@***.***>, Author ***@***.***> Subject: Re: [davidbolin/MetricGraph] joint models for metric data (Discussion #5) When you say "averages over lines" are these values constant over edges? Or can you have different values on a same edge? — Reply to this email directly, view it on GitHub<#5 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ANIDL5KML4T2O4EUBNQFEBTZFVVKRAVCNFSM6AAAAABIWFKQNSVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4TMNRQGE3DQ>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[AlexandreBSimas]

Ahh, ok. Then it is a little bit more complicated indeed. I asked that because we now also have edge weights (that can be a data frame), but they need to be constant values along the edges.

[smar-git]

Writing our own cgeneric was the first option…then we realized that we could then only fit models with alpha = 1 so we thought we could use your implementation instead…. By the way…I get a message saying that only alpha = 1 is implemented. Is this correct? From: Alexandre B. Simas ***@***.***> Date: Tuesday, 4 June 2024 at 09:28 To: davidbolin/MetricGraph ***@***.***> Cc: Sara Martino ***@***.***>, Author ***@***.***> Subject: Re: [davidbolin/MetricGraph] joint models for metric data (Discussion #5) How is bru overwriting the matrix? Which model are you passing there, the same spde_model? If you create a new cgeneric class you can create your own fm_basis in inlabru, and it would not overwrite it. — Reply to this email directly, view it on GitHub<#5 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ANIDL5N46VQAFTQHYVPE32DZFVUBLAVCNFSM6AAAAABIWFKQNSVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4TMNRQGAYTG>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[AlexandreBSimas]

Ahh, as I mentioned in the first reply, we only have alpha=1 for now, but we will be implementing alpha=2 very soon. We will, actually, also implement a rational approximation (without finite elements) that you will be able to estimate alpha as well. I will let you know here when we have them available.

I believe the best way for you is to really create a cgeneric (you can use our cgeneric and simply overwrite the class) for this model indeed.

[AlexandreBSimas]

It is also important to note for the alpha=2 case, that the precision also includes the derivatives, so the A matrix in that case will need to be more special, since it will have to take the average of the observations, but exclude the derivatives. You should also be able to do the same for the rational implementation we will have. I believe in one month or so we should have both the alpha=2 implementation and the rational approximation implementation.

[AlexandreBSimas]

So, to summarise. I really believe you can simply create a wrapper that uses our graph_spde() and changes the class to a different class. Then, you can create those ibm_ functions for inlabru, in which you can use the same as ours for everything except ibm_jacobian, where you can add you A matrix creator (all of this in a package as you need them as S3 methods). Then, you should have it working in inlabru for alpha = 1.

As soon as we have alpha=2, and the rational approximation (without fem) implemented, I will let you know here, and you can include those in your wrapper.

Also, if you have any difficulty in having it work, you can let me know here and I will do my best to help you.

[davidbolin]

For the exact models, we indeed only have alpha=1 implemented in the inla interface as @vpnsctl mentioned, but you can use the rspde.metric_graph function to fit alpha=2 models in inla using FEM approximations. See

https://davidbolin.github.io/MetricGraph/articles/fem_models.html#using-the-r-inla-implementation

In the function, you can provide nu = 1.5 if you want to fix alpha = 2.

[AlexandreBSimas]

Yes. She just needs to be careful that here the A matrix is different from the exact models. So they need a bit of care when implementing their model here.

[smar-git]

Ok I have tried that data1 = graph$process_data(data = df_graph) data2 = graph_bru_process_data(data1) but data1 and data2 seem to be identical so I don’t see that something was added…did I missed something?

str(data1)

mtrc_gr_ [30 × 7] (S3: metric_graph_data/tbl_df/tbl/data.frame) $ y : num [1:30] -1.41 -5.17 -1.85 4.71 1.16 ... $ covariate : num [1:30] -1.65 -0.423 0.82 0.474 0.964 ... $ .edge_number : num [1:30] 5 6 8 14 17 18 23 28 29 31 ... $ .distance_on_edge: num [1:30] 0 0 0 0 0 0 0 0 0 0 ... $ .group : num [1:30] 1 1 1 1 1 1 1 1 1 1 ... $ .coord_x : num [1:30] 0.2 0.25 0.35 0.65 0.8 0.85 0 0 0 0 ... $ .coord_y : num [1:30] 0 0 0 0 0 0 0.1 0.35 0.4 0.5 ...

str(data2)

mtrc_gr_ [30 × 7] (S3: metric_graph_data/tbl_df/tbl/data.frame) $ y : num [1:30] -1.41 -5.17 -1.85 4.71 1.16 ... $ covariate : num [1:30] -1.65 -0.423 0.82 0.474 0.964 ... $ .edge_number : num [1:30] 5 6 8 14 17 18 23 28 29 31 ... $ .distance_on_edge: num [1:30] 0 0 0 0 0 0 0 0 0 0 ... $ .group : num [1:30] 1 1 1 1 1 1 1 1 1 1 ... $ .coord_x : num [1:30] 0.2 0.25 0.35 0.65 0.8 0.85 0 0 0 0 ... $ .coord_y : num [1:30] 0 0 0 0 0 0 0.1 0.35 0.4 0.5 ... From: Alexandre B. Simas ***@***.***> Date: Tuesday, 4 June 2024 at 15:43 To: davidbolin/MetricGraph ***@***.***> Cc: Sara Martino ***@***.***>, Author ***@***.***> Subject: Re: [davidbolin/MetricGraph] joint models for metric data (Discussion #5) Yes, exactly. You can either add the location manually, or use the function (that essentially only does it): graph_bru_process_data() You can see the content of the function here: graph_bru_process_data <- function(data, edge_number = "edge_number", distance_on_edge = "distance_on_edge", loc = "loc"){ data[[loc]] <- cbind(data[[edge_number]], data[[distance_on_edge]]) data[[edge_number]] <- NULL data[[distance_on_edge]] <- NULL return(data) } — Reply to this email directly, view it on GitHub<#5 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ANIDL5M5EWKZQTQE5M6YVPLZFW77XAVCNFSM6AAAAABIWFKQNSVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4TMNRVGEZTG>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[AlexandreBSimas]

Ahh, since you are using the result of process_data as input, you need to supply edge_number = ".edge_number" and distance_on_edge = ".distance_on_edge" as arguments to graph_bru_process_data. I will update the function graph_bru_process_data now to automatically recognize that you are giving a metric_graph_data object, so you wont need to do it in the future.

[smar-git]

Got it! Thanks a lot From: Alexandre B. Simas ***@***.***> Date: Tuesday, 4 June 2024 at 15:55 To: davidbolin/MetricGraph ***@***.***> Cc: Sara Martino ***@***.***>, Author ***@***.***> Subject: Re: [davidbolin/MetricGraph] joint models for metric data (Discussion #5) Ahh, since you are using the result of process_data as input, you need to supply the edge_number = ".edge_number" and distance_on_edge = ".distance_on_edge" as arguments to graph_bru_process_data. I will update it now to automatically recognize that you are giving a metric_graph_data object, so you wont need to do it. — Reply to this email directly, view it on GitHub<#5 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ANIDL5I64DTBORDEYSYPYFTZFXBMFAVCNFSM6AAAAABIWFKQNSVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4TMNRVGMYTE>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[smar-git]

One more question. If I do not add any data to the graph when trying to build a spde model using spde_model <- graph_spde(graph, alpha =1) I get the following error: Error in abs(t) : non-numeric argument to mathematical function In addition: Warning message: In self$get_PtE() : There is no data! Should I just add a “fake” data then? sara From: Alexandre B. Simas ***@***.***> Date: Tuesday, 4 June 2024 at 15:55 To: davidbolin/MetricGraph ***@***.***> Cc: Sara Martino ***@***.***>, Author ***@***.***> Subject: Re: [davidbolin/MetricGraph] joint models for metric data (Discussion #5) Ahh, since you are using the result of process_data as input, you need to supply the edge_number = ".edge_number" and distance_on_edge = ".distance_on_edge" as arguments to graph_bru_process_data. I will update it now to automatically recognize that you are giving a metric_graph_data object, so you wont need to do it. — Reply to this email directly, view it on GitHub<#5 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ANIDL5I64DTBORDEYSYPYFTZFXBMFAVCNFSM6AAAAABIWFKQNSVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4TMNRVGMYTE>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[AlexandreBSimas]

Hi Sara,

Can you install the most recent version and try again (without adding any data)?

To install the most recent version you can run:

remotes::install_github("davidbolin/metricgraph", ref = "devel")

[smar-git]

Hi Now

spde_model <- graph_spde(graph, alpha =1)

Gives no error, but then I get that

data2 = graph$process_data(data = df_graph[c(1:10),]) data2 = graph_bru_process_data(data2,

+ edge_number = ".edge_number", + distance_on_edge = ".distance_on_edge")

spde_model <- graph_spde(graph, alpha =1) cmp <- ~ -1 + Intercept(1) + cov(covariate, model = "linear") + field(loc,

+ model = spde_model)

lik1 = like(formula = y ~.,

+ data=data2)

spde_bru_fit <- bru(cmp, lik1)

Error in 1:nrow(pte_tmp) : argument of length 0 In addition: Warning message: In model$graph_spde$get_PtE() : There is no data! From: Alexandre B. Simas ***@***.***> Date: Tuesday, 4 June 2024 at 17:45 To: davidbolin/MetricGraph ***@***.***> Cc: Sara Martino ***@***.***>, Author ***@***.***> Subject: Re: [davidbolin/MetricGraph] joint models for metric data (Discussion #5) Hi Sara, Can you install the most recent version and try again (without adding any data)? To install the most recent version you can run: remotes::install_github("davidbolin/metricgraph", ref = "devel") — Reply to this email directly, view it on GitHub<#5 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ANIDL5I5J7UTGGY2M3IGPFLZFXOIXAVCNFSM6AAAAABIWFKQNSVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4TMNRWGY4TQ>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[AlexandreBSimas]

Ahh, what happens is that if you want to use our A matrix for our cgeneric model, the data really needs to be inside the graph. (It is impossible with our construction to build the A matrix otherwise).

Basically our current constructor of A matrix identifies the input locations on the data inside the graph.

However, If you have an alternative constructor for A matrix, you should be able to do that thing I told you. To create an alternative cgeneric (basically changing the class of our cgeneric model) and create a different ibm_jacobian for such class.

alternatively, if you want to use our A matrix with more than one likelihood then the only way is really to add all observations, and pass the complete dataset.

[smar-git]

Ok I see… But adding all observations will not work as in that case (I guess) they will all have the same error/noise parameter while what I want is them to have two different error model… What would happen if I just add a NA on the graph and then use the data I create through graph$process_data To the like() function? sara From: Alexandre B. Simas ***@***.***> Date: Tuesday, 4 June 2024 at 19:33 To: davidbolin/MetricGraph ***@***.***> Cc: Sara Martino ***@***.***>, Author ***@***.***> Subject: Re: [davidbolin/MetricGraph] joint models for metric data (Discussion #5) Ahh, what happens is that if you want to use our A matrix for our cgeneric model, the data really needs to be inside the graph. (It is impossible with our construction to build the A matrix otherwise). Basically our current constructor of A matrix identifies the input locations on the data inside the graph. However, If you have an alternative constructor for A matrix, you should be able to do that thing I told you. To create an alternative cgeneric (basically changing the class of our cgeneric model) and create a different ibm_jacobian for such class. alternatively, if you want to use our A matrix with more than one likelihood then the only way is really to add all observations, and pass the complete dataset. — Reply to this email directly, view it on GitHub<#5 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ANIDL5PVWTOGVMNZHKZ7HETZFX26LAVCNFSM6AAAAABIWFKQNSVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4TMNRXHA2TS>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[AlexandreBSimas]

I believe it should work. Since what we need is really that all the positions in which we have observations exist on the graph. (Internally, what we do is that we turn the observations into vertices, since our model is invariant under addition/removal of vertices of degree 2, and then we subset the observations we want to use).

Another possibility you could do, and I believe would also work, is to use the group argument. You can create one column group where for group==1 you have the data for likelihood 1 and for group==2 you have the data for likelihood 2.

[smar-git]

Mmm no, this does not work either… I get the error

spde_bru_fit <- bru(cmp, lik1)

Error in ..subscript.2ary(x, l[[1L]], l[[2L]], drop = drop[1L]) : NA subscripts in x[i,j] not supported for 'x' inheriting from sparseMatrix I am attaching a code you can run in case … Also the group idea I don’t think will do what I want… From: Alexandre B. Simas ***@***.***> Date: Tuesday, 4 June 2024 at 19:41 To: davidbolin/MetricGraph ***@***.***> Cc: Sara Martino ***@***.***>, Author ***@***.***> Subject: Re: [davidbolin/MetricGraph] joint models for metric data (Discussion #5) I believe it should work. Since what we need is really that all the positions in which we have observations exist on the graph. (Internally, what we do is that we turn the observations into vertices, since our model is invariant under addition/removal of vertices of degree 2, and then we subset the observations we want to use). Another possibility you could do, and I believe would also work, is to use the group argument. You can create one column group where for group==1 you have the data for likelihood 1 and for group==2 you have the data for likelihood 2. — Reply to this email directly, view it on GitHub<#5 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ANIDL5KBPRN25S5RBSELW6LZFX343AVCNFSM6AAAAABIWFKQNSVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4TMNRXHEZTS>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[AlexandreBSimas]

Hi Sara,

I did not receive any attachment. I guess you attached on email? Maybe you can directly attach on github (and not on email)?

With respect to the group, it should really work. Let me clarify to you what the group does internally, so it becomes clearer. When we add observations from more than 1 different groups, (either directly adding all observations, or by adding sequentially), all the locations from all the groups are "stored" for each group. More precisely, if there is a location in group 2 that is not a location in group 1, it will add a location in group 1 with NA, and vice-versa.

So, the PtE (that gives the relative position on edges and edge numbers for all the locations on the graph) will contain all the locations across all groups. This means the A matrix will be created from all the available locations, and once you pass data from group 1, it will subset the positions for group 1 to create A for that case, and similarly for group 2.

If, even taking that into consideration you believe the group should not work, I would like to know why, because it should not be any different from the original idea to add NA (because this is literally what is being done internally).

[smar-git]

Hi I have tried to upload an R file on github but it seems like it is only possible to upload pdf or jpg, not text files. As for the group, is this the same group as for the inla() function? Because in that case I can understand that you can have two errors..but that would be another random effect, not another likelihood, so you could not use the same trick if instead of two gaussian likelihoods you would have, say, a gausssian and a poisson likelihood…am I wrong? From: Alexandre B. Simas ***@***.***> Date: Wednesday, 5 June 2024 at 11:52 To: davidbolin/MetricGraph ***@***.***> Cc: Sara Martino ***@***.***>, Author ***@***.***> Subject: Re: [davidbolin/MetricGraph] joint models for metric data (Discussion #5) Hi Sara, I did not receive any attachment. I guess you attached on email? Maybe you can directly attach on github (and not on email)? With respect to the group, it should really work. Let me clarify to you what the group does internally, so it becomes clearer. When we add observations from more than 1 different groups, (either directly adding all observations, or by adding sequentially), all the locations from all the groups are "stored" for each group. More precisely, if there is a location in group 2 that is not a location in group 1, it will add a location in group 1 with NA, and vice-versa. So, the PtE (that gives the relative position on edges and edge numbers for all the locations on the graph) will contain all the locations across all groups. This means the A matrix will be created from all the available locations, and once you pass data from group 1, it will subset the positions for group 1 to create A for that case, and similarly for group 2. If, even taking that into consideration you believe the group should not work, I would like to know why, because it should not be any different from the original idea to add NA (because this is literally what is being done internally). — Reply to this email directly, view it on GitHub<#5 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ANIDL5NCLMGPR27SQYNV6JDZF3NV3AVCNFSM6AAAAABIWFKQNSVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4TMNZWGAYDS>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[AlexandreBSimas]

There is also another possibility that might be more convenient, and maybe reduce the amount of data processing. If you look at how we handle observations on metric graphs internally, you can see that we store as list. This means you can add as "observation" any object for each location, in particular it can be a matrix for each location, or a vector, etc. So, for example, you should be able to add y as a matrix for each location (so that you have y as cbind(y, NA), for the first likelihood, or y as cbind(NA,y) for the second).

[AlexandreBSimas]

Can you write the model which you want to fit with multiple likelihoods? And maybe add R code?

From what I see, you can attach .md files. Maybe you can simply rename an R file as .md file and attach?

[smar-git]

here is the code
Report.md

[smar-git]

Hi Alexandre,
thanks a lot for all the help.
To be honest I am not sure i understand how to use the group option nor who i should preprocess the data.
My model is quite simple: basically i have two data sources, say y1 and y2 that are noisy observation of the same process, say eta. So that y1N(eta,sigma1) and y2N(eta,sigma2). And eta is a GF of a metric graph. The observation don't have to be at the same location on the graph (although they might).
The code i sent you earlier tries to do exactly this but without success...
bests
Sara

[AlexandreBSimas]

Hi Sara,

I implemented the alpha=2 case in cgeneric, so you can fit in inla. I will now check that everything is ok with the case alpha=2 (I will then add the alpha=2 case to inlabru, and adjust the predict, but should be fast).

Then, I will prepare a vignette with multiple likelihoods for you. I will use your model to illustrate. I really understand that the internal data structure of the metric graph is a bit confusing.

In the same spirit, I discussed with David a bit and he suggested (and I agreed) that we should add an "advanced data manipulation" section on the data manipulation vignette. So I will get your example of multiple likelihoods as a motivation for the advanced data manipulation section as well, to show how one can manually handle such situations. For the multiple likelihoods my goal is to show how to fit there using the data helper functions (that I will update).

Should not take long. I will let you know when things are ready, so you can take a look.

[smar-git]

Thanks a lot Alexandre, that sounds really useful! From: Alexandre B. Simas ***@***.***> Date: Friday, 7 June 2024 at 11:25 To: davidbolin/MetricGraph ***@***.***> Cc: Sara Martino ***@***.***>, Author ***@***.***> Subject: Re: [davidbolin/MetricGraph] joint models for metric data (Discussion #5) Hi Sara, I implemented the alpha=2 case in cgeneric, so you can fit in inla. I will now check that everything is ok with the case alpha=2 (I will then add the alpha=2 case to inlabru, and adjust the predict, but should be fast). Then, I will prepare a vignette with multiple likelihoods for you. I will use your model to illustrate. I really understand that the internal data structure of the metric graph is a bit confusing. In the same spirit, I discussed with David a bit and he suggested (and I agreed) that we should add an "advanced data manipulation" section on the data manipulation vignette. So I will get your example of multiple likelihoods as a motivation for the advanced data manipulation section as well, to show how one can manually handle such situations. For the multiple likelihoods my goal is to show how to fit there using the data helper functions (that I will update). Should not take long. I will let you know when things are ready, so you can take a look. — Reply to this email directly, view it on GitHub<#5 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ANIDL5JWA2TRIRF344EY4WDZGF4BPAVCNFSM6AAAAABIWFKQNSVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4TMOJZG42DC>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[AlexandreBSimas]

Since you will be using, and we really needed to add this alpha=2 soon, I decided to directly implement the alpha=2 case in INLA. It is now available. So, you should be able to use it now. See the details in the discussion for that (I opened a different discussion). For the moment, it will need a special "way" to be able to run it (you will need to install our rSPDE package from source), but soon enough will be trivial (when Håvard releases a new update for INLA, it will be there in inla, so there will be no need to install rSPDE from source, then).

[smar-git]

Dear David and Alexandre I have found a way to implement the mode where the data are not observations on a point ( as in your example) but averages over lines using the new A.local argument that Havard has added to the f() function. We still do not manage to fit a joint model where we have two likelihoods with some shared components. This is the problem where you suggested to use the “group” option. But I still do not understand how to implement this. You saied that you were going to write some more documentation and I wanted to ask if you manage to do that. Thanks a lot Sara