Add Support for Importing Bibtex Files

[josepaiva94]

I have a list of relevant publications in a bibtex file. How can I start snowball process from such list?

[blochberger]

You could use bibtexparser to process the file in Python and create the necessary objects (Author and Publication), see for example:

sokman/sok/management/commands/dblpimport.py

Lines 446 to 484 in c784860

	for result in results:
	# Add authors to database
	authors: List[Author] = []
	for name in result.authors:
	author, created = Author.objects.get_or_create(name=name)
	if created:
	self.log_success(f"Added author: {author}")
	else:
	self.log_info(f"Author '{author}' alreay known")
	authors.append(author)
	# Add publication to database
	publication, created = Publication.objects.get_or_create(
	cite_key=result.cite_key,
	title=result.title,
	year=result.year,
	peer_reviewed=result.is_peer_reviewed,
	first_page=result.first_page,
	last_page=result.last_page,
	doi=result.doi,
	)
	if created:
	self.log_success(f"Added publication: {publication}")
	else:
	self.log_info(f"Publication '{publication}' already known")
	publications.append(publication)
	# Assign authors
	for position, author in enumerate(authors):
	publication_author, created = PublicationAuthor.objects.get_or_create(
	author=author,
	publication=publication,
	position=position,
	)
	if created:
	self.log_success(f"Assigned author '{author}' to publication '{publication}' at position {position}")
	else:
	self.log_info(f"Author '{author}' already assigned to publication '{publication}' at position '{position}'")

If you already use DBLP cite keys in your file, you can simply import the publications with

./manage.py dblpimport 'DBLP:conf/ease/PetersenFMM08' 'DBLP:conf/ccs/EgeleBFK13'

You can provide as many cite keys in the single command. Add --use-api flag, if you do not have a DBLP dump and want to fetch a live version from the server. But be aware that a request is made for each key provided.

Note that the Semantic Search API integration that is used for snowballing works based on the DOI (or Semantic Search paper ID). If you have a DOI in your bibtex file, it should work out of the box, else you might need to assign the IDs accordingly. You could search DBLP for the publication title to semi-automatically identify related DBLP entries (or more specifically DOIs). You could do the same with the Semantic Search API.

[josepaiva94]

I have converted the bibtex entries into an SQL insert statement on sok_publications. Then, I run ./manage.py repair and I can now ./manage.py snowball :-)

[blochberger]

I think the use-case is still interesting and is worth supporting. Hence, I keep the issue open as a reminder.

[josepaiva94]

The small script for anyone facing the same use-case https://gist.github.com/josepaiva94/c91834935923e8394aa19ed766d8fa51 (DOIs are mandatory!)