Memory leaks in unifyfsd

[rgmiller]

System information

Type	Version/Name
Operating System	Linux
OS Version	Ubuntu 20.04
Architecture	x86_64
UnifyFS Version	dev branch

Describe the problem you're observing

I've been running unifyfsd with valgrind, and it (valgrind) is reporting a number of memory leaks. Most of the errors seem to be pretty benign: allocate some space at program startup and forget to free it before the program ends. It looks like some, though, do have the potential to leak memory continuously during a run.

Describe how to reproduce the problem

I've been using valgrind --leak-check=full --show-leak-kinds=all, so it's reporting pretty much everything.

Include any warning or errors or relevant debugging data

Below is a partial list of the specific errors I've confirmed so far:

• Never calling unifyfs_config_fini() to free up configuration variables
• Never calling ABT_finalize() to free up the Argobots data
  • Fixed by use ABT after calling margo_init #659
• Not calling ABT_mutex_free() to free up the app_configs_abt_sync mutex
• Not freeing the elems pointer when cleaning up arraylist_t structs
  • Fixed by ea2f5f5
• Global server_info_t array allocated but never freed
  • margo_server.c, lines 37 & 506
• When the request manager struct (reqmgr_thrd_t) is freed, the client_reqs array list it (probably) allocated is not freed
  • Happens every time a request manager thread is cleaned up
  • Same is true for the client_callbacks array list
• unifyfs_inode_tree_clear() deletes inode structs without first deleting the allocated members of those structs
• Request manager thread doesn't free the reqs_sync mutex when the thread exits
• fattr.filename in create_mountpoint_dir() is allocated using strdup(), but never freed.
  • This one is going to be tricky because there appears to be no good place to do the free(). See the comment below for details.

[CamStan]

• Never calling ABT_finalize() to free up the Argobots data

Will #659 solve this one as UnifyFS will no longer be directly calling ABT_init()?

[rgmiller]

Will #659 solve this one as UnifyFS will no longer be directly calling ABT_init()?

I think so. I'll update the list above.

[rgmiller]

Fixing the fattr.filename leak is going to be tricky because there appears to be no good place to free it.

Inside of create_mountpoint_dir(), the fattr struct gets assigned to a unifyfs_metaset_in_t member. This metaset is then assigned to a client_rpc_req_t member. The request is then passed to rm_submit_client_rpc_request() where it is added to an arraylist. The appropriate request manager thread eventually processes this arraylist in rm_process_client_requests().

The arraylist is freed at the bottom of rm_process_client_requests(). This is where the trouble lies: arraylist_free() can and does call free() on each element. However, arraylist is a generic container; its elements are just void *. There's no way for arraylist_free() to know that the element is a client_rpc_req_t that contains a unifyfs_metaset_in_t that contains a unifyfs_file_attr_t who's filename member needs to be explicitly freed.

Ok, that initial analysis wasn't correct. The unifyfs_metaset_in_t struct is being freed in process_metaset_rpc(). However, it looks like that same function overwrites the filename with a duplicate of itself?!? I'm not sure what was intended there, but it seems that's the source of the memory leak.

[rgmiller]

Everything has been merged. Closing the issue.