addEventAsync() issue under load

[yoanntrouillet]

We are having some issues with the Keen.io java client we are pushing a lot of events. those event are leagcy event and we are overriding the timestamp value of each event. We using the addEventAsync() method.

what we observed is that under load this does not work as expected and events are dropped.

    protected Executor getDefaultPublishExecutor() throws Exception {
        int procCount = Runtime.getRuntime().availableProcessors();
        return Executors.newFixedThreadPool(procCount);
    }

which returns a instances of ThreadPoolExecutor as per
public static ExecutorService newFixedThreadPool(int nThreads) {
return new ThreadPoolExecutor(nThreads, nThreads, 0L, TimeUnit.MILLISECONDS, new LinkedBlockingQueue());
}

We think the TreadPoolExecutor implementation drops requests when all the threads are busy doing pushing event to keen.io

We cannot use sendQueuedEventsAsync() as we need to control the timestamp value of the events ?

I don't really understand (didn't look into it) why we cannot specified (override) the timestamp of the events when queuing them ?

any other suggestion on how we could this ?

[Geeber]

Sorry that you're having trouble. I'm not sure exactly why you'd be getting dropped events from the synchronous code, but you should be able to use queueEvent and set the keen.timestamp. Your problem may have been trying to set keen.timestamp in your normal event map, rather than in the keenProperties map. See this override of the queueEvent method:

https://github.com/keenlabs/KeenClient-Java/blob/master/core/src/main/java/io/keen/client/java/KeenClient.java#L263

Unfortunately this isn't very clearly documented, but if you override keen.timestamp in the keenProperties map it should work. Could you give that a try and let me know if it works?

[SteeleParker]

Hi Geeber,

We had success with the queueEvent and were able to successfully set the keen.timestamp through the keen properties. Thanks for that tip. I had understood the documentation to mean that keen.timestamp was not supported in queueEvent.

Unfortunately it appears that there is no way to retrieve the result of sendQueuedEvents through the callback argument in order for us to collect and store the result of each event in the queue, leaving us in a similar position to before.

Is there any currently undocumented feature which would allow us to retrieve the JSON result. At the moment our approach would be to extend KeenClient and Override handleAddEventsResponse, calling super.handleAddEventsResponse( [args] ) as it already has access to the server response, however this is a private method.

[Geeber]

Unfortunately we don't currently provide access to the results. Out of curiosity, why do you need to see them? The SDK handles processing the results and retrying if there were transient failures, such as network connectivity issues. If you explain your specific application requirements a bit more, I might be able to suggest an alternative approach (or convince you that you don't need to worry about it).

[SteeleParker]

We would like to collect and persist any exceptions caused, alongside the DB record which caused it for later analysis and improvement of the quality of data we are pushing without needed to enable the logging in a production environment.

For example, we recently discovered whilst reviewing near-to-production data in a development environment that some of the pushed events are not appearing in Keen.io because they attempted to geocode an IP address field which had two IP addresses contained within it in the format of x.x.x.x, x.x.x.x instead of the expected x.x.x.x with the ipToGeo plugin. As it stands, we currently have no way of detecting this error, persisting it and reviewing it later.

We'd also like to only mark each record as sent once we have flushed the queue and received a success response for each event but due to the amount of data we need to push, we cannot send events individually