-
Notifications
You must be signed in to change notification settings - Fork 108
Accepted proposals for Google Summer of Code 2016
On this section you will find a list of the accepted proposals for Google Summer of Code 2016
- Student: Student name
- Mentors: Mentors list
- Student: Vladimir Cernov [[email protected]]
- Mentors: Bruno Dilly, ceolin, cmarcelo
I want to contribute to Soletta by implementing one of the proposed ideas, namely “Implement Go bindings”. This implies providing a development interface accessible from the Go programming language, thus enabling Go developers out there to use the Soletta framework to build applications for their devices. With bindings for NodeJS and Python already under development, adding Go would further enhance Soletta library.
Though Go rose to popularity only in recent years, it becomes widely used in all kind of projects, ranging from web [9] to IoT [8]. Its strengths include performance, stability, being open source and supported by a large community, a design focused on effectiveness and efficiency and many available development tools. Considering these facts, I believe providing Go bindings would be a real asset for Soletta.
Below I tried to structure the contributions to Soletta as part of GSoC program into several core steps:
- Determination of Soletta parts that need to be interfaced with Go
- Design of Go interface
- Build system integration
- Bindings for Soletta mainloop
- Bindings for flow mechanism
- Bindings for other parts of Soletta API
- Implementation of samples in Go
- Writing tests for Go bindings
- Writing documentation
I tried to list them in a logical, chronological order, but this is not really true for the tests implementation, as those need to be written incrementally throughout the GSoC project period.
Following is an approximate schedule for accomplishing important milestones. I provided an estimated end date for each task along with their expected duration. There is a long a period of time until the coding period starts (proposal selection period + community bonding period), which I plan to use to get familiar with particular Soletta components relevant to the project, contributing to Soletta (there were added recently a lot issues with this purpose). I expect after this period to have a very clear understanding of how the project should be developed.
- 22 May End of bonding period, start of GSoC program
- 5 June API design (2 weeks)
- 12 June Build system integration (1 week)
- 19 June Mainloop (1 week)
- 27 June Midterm evaluation
- 24 July Flow (4 weeks)
- 7 August Other components (2 weeks)
- 14 August Samples, tests and documentation (1 week)
- 23 August Final evaluation
- Post GSoC Maintenance of the Go bindings
I left out 2 weeks in the timeline (one pre and one post midterm) to accommodate delays or other unpredictable events. If that won’t be the case, they will be used as additional time for project implementation.
For the GSoC program I plan to allocate at least 30 hours per week, in flexible manner. My timezone is GMT+2 (EET). I consider the time difference won’t be an issue, especially since I was always able to effectively communicate with Soletta developers prior and during GSoC application period.
I have considered multiple approaches to implement Go bindings.
Swig tool suite allows to specify a “glowing” interface in a “.i” file. One can write wrappers for C or C++ code that will be accessible from Go. Swig can also be used to write bindings for other programming languages, so it would have been useful if Soletta would have been providing all bindings (nodejs, python) using swig, however that is not the case.
Some downsides of this approach would be that there are some tricky corner cases when dealing with memory allocation and require a separate tool chain (additional build dependency).
cgo comes with the Go toolchain and feels like it is built-in. It enables linking against C programming language, Go to reference C (writing extension modules for Go applications in C) and C to reference Go (e.g. create Go interface for C library). It appears to be straightforward to use, while providing all the necessary support for writing bindings.
The downsides of this approach that I found include the fact that it only interfaces with C code (not applicable for Soletta, since it is written in C) and there is somewhat limited documentation (though I believe it is sufficient) on cgo.
In conclusion, I think that the CGo approach is more suitable to accomplish the goal of the project. To illustrate this, I prepared a Proof of Concept [2] (which I encourage to take a look at), as part of the proposal that shows the basic workflow for binding Soletta. Though it is a minimalistic example, it works and, aside from illustrating a simple usage on how a user could develop in Go for Soletta, it shows that cgo is a feasible approach:
C sample:
static int data;
static bool
on_timeout_cb(void *data)
{
printf("data = %d\n", *((int *) data));
(*((int *) data))++;
return true;
}
int main(int argc, char **argv)
{
sol_init();
sol_timeout_add(1000, &on_timeout_cb, &data);
sol_run();
sol_shutdown();
return 0;
}
Equivalent Go sample with same functionality:
func on_timeout_cb(data *int) int {
println("data =", *data)
(*data)++
return 1
}
var data int
func main() {
s := soletta.Soletta{}
s.Start()
s.AddTimeout(1000, on_timeout_cb, &data)
s.Run()
s.Stop()
}
Just as a note, binding sol_timeout_add is this just a demonstrative example, to test cgo features, as the function doesn’t require bindings because it has a Go equivalent (channels and select).
One goal of the project (also specified in the ideas page) is to design a Go-like interface, that is to make use of existing libraries and programming idioms over Soletta. This require a research on which Soletta modules have or do not have a Go alternative, and a comparison in the cases where the Go equivalent is present. Since Soletta mainloop is core, it has to be wrapped in a way and the same apply for the flow mechanism, which doesn’t really have an equivalent in Go.
Ideally, the unavailable Soletta C-API [11] from Go should be covered as much as possible. However, parts of the API, such as data structures, timeouts, etc. will not be binded, since there are Go specific features such as coroutines and channels, for example, that accomplish the same thing.
The project go-qml [21] has a similar structure to what we want to achieve, so it can be used to find ideas and approaches for achieving the goals of this proposal.
The Go bindings need to be built (compiled) somehow, in a way that would impact the end-user in a minimum way. I see 2 ways of accomplishing (not entirely disjunct) this: The Go bindings component gets a new entry in the current Soletta build system (makefiles, configuration files, etc), similarly to how nodejs and Python bindings are treated. Make use of Go get command and install the bindings as a third-party package, e.g. from a git repository. One could install Soletta package (or build from source), and then install the bindings package separately. This increases the logic separation and modularity.
Either way, Go bindings code will live in a separate directory layout according to Go practice (e.g. $GOPATH/src/go-soletta-flow). The second way is preferable, since it has minimum impact on Soletta build system, however I think this approach may have some limitations if there would be a case where Soletta library would reference Go (which would increase the coupling).
As Caio Oliveira suggested, binding Soletta mainloop as is won’t have much value. So I plan to cover only the essential functionality from mainloop C API, and focus the effort on flow.
I think this can be done by directly wrapping the mainloop API in a Go interface. Some of the functions will have Go equivalents (sol_argc, sol_args_set, sol_argv, sol_timeout_add, sol_timeout_del), others (such as sol_idle_add, sol_idle_del, sol_mainloop_default_main) require passing callbacks from Go to C, which is possible using cgo. The API would be partially covered, with focus on those aspects that make more sense for a Go perspective.
Another approach is to replace (more specifically moving the mainloop implementation to Go bindings, and enabling it either via sol_mainloop_set_implemenation or sol_mainloop_source_add, or by including a separate source file like sol-mainloop-impl-posix.c) the mainloop mechanism entirely using Go channels and select statement, to provide asynchronous, event-driven programming. This approach seems more complex (since it involves C refering to Go code), and require careful examination when binding other components that inject events on the mainloop queue (e.g. flow nodes), however it allows more flexibility, should it be viable.
Binding the Soletta flow paradigm will be core for the summer project, as they have more value for Go developers. Though there is already a library (goflow [22]) that implements a dataflow and flow-based programming, it has ”quite a minimalistic” implementation as the author states. Soletta flow on the other hand has support for domain specific language, support for metatypes and access to existing nodes written in C.
Go bindings should cover at least the following: Loading of nodes Parsing Manually creating flows Create custom nodes A standalone container type for node interaction
In the beginning I think there should be designed a base interface which each node type will extend. There are a lot of node types [18], however there might not be the case to make individual bindings for each builtin node, as they can be loaded by name.
It is necessary to determine which Soletta components are available in Go in one form or another (built-in, native libraries or third-party libraries). Those who already have an alternative in Go don’t require bindings at all, while others may require just a thin wrapping layer. Following is a list of components that I tried to identify starting from Soletta C API and the status of their potential Go equivalent.
Types and Data types (Arena, Buffer, List, Vector, String table etc.) are mostly built-in. Same applies for Soletta utility functions. The Worker Thread API is provided by Go coroutines. Most MACROs are C specific and not applicable for Go language.
CoAP seems to be provided by go-coap [17] and canopus [16] HTTP is provided by Go package “net/http”. LWM2M seems to be provided by Betwixt [12] MAVLINK seems to be provided by go-mavlink [13] MQTT is provided by SurgeMQ [14] and mqtt [15] Network is provided by Go package “net”. OIC doesn’t seem to be available.
JSON is provided by Go package “encoding/json”. Message digest and Certificates are provided by Go packages “crypto” and “crypto/tls”. Logging is provided by Go package “log”.
Many Platform utilities are also provided by built-in Go packages, however they might need to be wrapped to be coupled with Soletta mainloop, since they use callback logic. I believe detaching them from mainloop and provide a pure Go implementation would also be viable, but requires more analysis and work.
Since a lot of Soletta components don’t require binding, the focus of the project will be on binding flow mechanism. If other components would be identified during the program to require binding, that will be covered in the available timeline.
It is also important to provide examples on how to use the newly created go bindings. Aside from the fact that these are helpful to end-users, they also help during the development process by acting as a test suite. Due to this reason, a part of the samples will be migrated to Go code during bindings implementation.
Since Soletta has a test suite, it would be great to have some tests for Go bindings too. The functional and tests with a higher level of abstraction would form a separate test suite (as is the case for nodejs bindings), but I think it would also be nice to have some unit tests where necessary.
I plan to use GoDoc [19] to generate documentation for Go bindings in the same way doxygen is currently used for C code. I would have used the same tool for Go to be consistent, however there doesn’t seem to be an easy or recommended way to do so.
Please note that writing tests and documentation is an ongoing process. Usually, the workflow I follow is design - implementation - testing - documentation, then proceed to next feature. In my experience, making granular, incremental steps leads to better code and fewer coding errors. So consider the entry in the timeline being “make sure there are enough samples, tests and documentation” instead of “write tests and documentation at the end of summer project”.
[1] http://monkey-project.com/
[2] https://github.com/kaspersky/soletta_gobindings_poc
[3] https://github.com/torvalds/linux/commit/ec9bed9d385fd094b20fa0809c50741710afdc74
[4] https://github.com/torvalds/linux/commit/c429df9df06af21912741d5a3848306328564262
[5] https://github.com/solettaproject/soletta/commits?author=kaspersky
[6] http://www.swig.org/Doc2.0/Go.html
[7] https://golang.org/cmd/cgo/
[8] http://gobot.io/
[9] http://thenewstack.io/a-survey-of-5-go-web-frameworks/
[10] https://github.com/monkey/monkey/commits?author=kaspersky
[11] http://solettaproject.github.io/docs/c-api/
[12] https://github.com/zubairhamed/betwixt
[13] https://github.com/ungerik/go-mavlink
[14] https://github.com/surgemq/surgemq
[15] https://github.com/jeffallen/mqtt
[16] https://github.com/zubairhamed/canopus
[17] https://github.com/dustin/go-coap
[18] http://solettaproject.github.io/docs/nodetypes/
[19] https://godoc.org/golang.org/x/tools/cmd/godoc