-
Notifications
You must be signed in to change notification settings - Fork 2
/
6-datasets.Rmd
51 lines (33 loc) · 6.94 KB
/
6-datasets.Rmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
# Datasets
<!--- last spell checked 2022-09-29--->
<!---update the datasets locations--->
In this chapter we introduce the patent datasets developed for the Open Source Patent Analytics Project as training sets for patent analytics. The datasets will be used in the walkthroughs. The datasets will grow over time but we will briefly introduce them and explain how to access them.
The datasets are housed at the project [GitHub repository](https://github.com/wipo-analytics/opensource-patent-analytics/tree/master/2_datasets). To download individual files click on the link and then select raw to download the file.
## The datasets
The datasets are intended to illustrate the range of possibilities for patent data including some of the challenges that may be encountered in cleaning and analysing patent data. They are also drawn from different sources.
### Core dataset
The `pizza_medium_clean` dataset is a precleaned version of the `pizza_medium` dataset. Specifically, the applicants and inventors field have already been cleaned along with corrupted characters and other common cleaning tasks. This makes it easier to work with the data.
This dataset is used in the Chapter on Tableau Public and Gephi. You can download the files for those chapters through the datasets homepage or directly as a zip file from this [link](https://osf.io/download/s8fa7/).
### Sample datasets
Almost everyone likes pizza and it is easy to search a patent database for the term "pizza". It is also an area of patent activity that encompasses a wide range of technologies such as pizza ovens, pizza boxes, pizza cutters and pizza toppings etc. It is therefore useful for demonstrating ways of interrogating patent data for particular topics.
1. `pizza_small` is a very small 26 row dataset created by downloading the first page of results from the [European Patent Office espacenet database](http://worldwide.espacenet.com/?locale=en_EP) for a smart search on "pizza". It's a quick and easy test dataset.
2. `pizza_medium` was created from a sample of data from a search of the [WIPO Patentscope database](http://www.wipo.int/patentscope/en/) for the term "pizza" and contains 9,996 rows of data. It is intended to illustrate the data format from Patentscope and to allow work on a medium sized dataset. Note that the format varies from the espacenet format and presents different challenges.
An important feature of Patentscope data from a statistical standpoint is that the field marked `publication_number` in the original data lacks a two letter kind code and is therefore an `application_number`.
### Patent Landscape Reports datasets
Three datasets are drawn from the [WIPO Patent Landscape Reports](http://www.wipo.int/patentscope/en/programs/patent_landscapes/). The datasets address different topics, present a variety of fields and formats and are different sizes. Each dataset is linked to a detailed patent landscape report that provides an insight into approaches to patent analytics.
1. `ewaste` presents the results of research for a [report](https://www.wipo.int/publications/en/details.jsp?id=388) on patent activity for electronic waste recycling and its implications for developing countries.
2. `solar_cooking` presents the data supporting a [report](https://www.wipo.int/publications/en/details.jsp?id=233) on technologies that use solar energy as the source for cooking and pasteurizing food.
3. `ritonavir` presents the data for a patent [report](https://www.wipo.int/publications/en/details.jsp?id=230&plang=EN) on patent activity for the HIV antiretroviral drug Ritonavir in the field of pharmaceuticals. The dataset illustrates specific activity around issues such as dosage and also the problem of 'evergreening' in patent activity.
4. [Artificial Intelligence](https://www.wipo.int/tech_trends/en/artificial_intelligence/). The [WIPO Technology Trends 2019 report](https://www.wipo.int/tech_trends/en/artificial_intelligence/) focused on artificial intelligence. The raw data used for the report is not available for direct download but the data for individual figures is available in a set of [multi-sheet excel tables](https://www.wipo.int/export/sites/www/tech_trends/en/artificial_intelligence/docs/techtrends_ai_data_2019.zip) for those interested. In addition, WIPO offers an [Artificial Intelligence Index](https://www.wipo.int/tech_trends/en/artificial_intelligence/patentscope.html) page with links from the search terms used to generate the report data in WIPO Patentscope. For example, the [Machine Learning link](https://patentscope.wipo.int/search/en/result.jsf?query=AItechniqueMachineLearning&sortOption=Relevance) will generate a new dataset with results. Users who have signed up for a free Patentscope account can then refine the search and download basic data fields for up to 10,000 records.
5. [Assistive Technology](https://www.wipo.int/publications/en/details.jsp?id=4541). The [WIPO Technology Trends 2021 report](https://www.wipo.int/edocs/pubdocs/en/wipo_pub_1055_2021.pdf) focused on assistive technologies. The [report]((https://www.wipo.int/edocs/pubdocs/en/wipo_pub_1055_2021.pdf)) includes two datasets in Excel.
- [Conventional Assistive Technology Dataset](https://www.wipo.int/edocs/pubdocs/en/wipo_pub_1055_2021-tech1.xlsx)
- [Emerging Assistive Technology Dataset](https://www.wipo.int/edocs/pubdocs/en/wipo_pub_1055_2021-tech2.xlsx)
<!--- ### Packaged Datasets
We have created two packages in R that can be downloaded for use in R or Python.
1. [Drones](https://github.com/wipo-analytics/drones)
2. Women Inventors. Forthcoming in 2022.
### Other datasets
1. `wipo` is a single Excel sheet of data on trends in patent applications and growth rates from the [WIPO World Intellectual Property Indicators - 2014 Edition](http://www.wipo.int/ipstats/en/wipi/). The data is used for simple graphing in tools such as R and illustrates the need to skip rows when reading data into analytics tools. More recent data from WIPO annual indicators reports is available for individual reports on the [WIPO Indicators page](https://www.wipo.int/publications/en/series/index.jsp?id=37). For example, the [WIPO Indicators 2021 report](https://www.wipo.int/publications/en/details.jsp?id=4571&plang=EN) page includes a set of multi-sheet excel workbooks with the data used to generate the figures for the indicators set.
2. `WIPO_sequence_data`. This dataset contains a small sample of the sequence data from the year 2000 available free of charge from the [WIPO Patentscope database](https://patentscope.wipo.int/search/en/sequences.jsf). This dataset can be used to explore analysis of patent sequence data.
3. [Synthetic biology](https://github.com/poldham/opensource-patent-analytics/tree/master/2_datasets/synbio_patents). This is a sample of data from Thomson Innovation developed by Paul Oldham for research on patent activity involving synthetic biology. The data has been extensively cleaned in VantagePoint from Search Technology Inc. and is intended to illustrate the use of data from a commercial patent database.
--->