-
Notifications
You must be signed in to change notification settings - Fork 2
/
2016_acl_doclabel.tex
146 lines (103 loc) · 3.94 KB
/
2016_acl_doclabel.tex
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
\documentclass[11pt]{article}
\usepackage{style/acl2016}
\usepackage{titlesec}
\usepackage{graphicx}
\usepackage{amsmath}
%\usepackage{subfig}
\usepackage{times}
\usepackage{url}
\usepackage{latexsym}
\usepackage{breakurl}
\usepackage{subcaption}
\makeatletter\newcommand{\@BIBLABEL}{\@emptybiblabel}\newcommand{\@emptybiblabel}[1]{}\makeatother
\def\imagebox#1#2{\vtop to #1{\null\hbox{#2}\vfill}}
\usepackage[bookmarks=false,
pdfpagelabels=false,
hyperfootnotes=false,
hyperindex=false,
pageanchor=false,
]{hyperref}
\makeatletter
\let\saved@hyper@linkurl\hyper@linkurl
\let\saved@hyper@link@\hyper@link@
\AtBeginDocument{
\NoHyper
}
\aclfinalcopy
\newif\ifcomment\commentfalse
\input{style/preamble}
\newcommand{\name}[0]{\abr{alto}}
\addtolength\titlebox{.5in}
\newcommand{\sect}[1]{\label{sec:#1} \input{sections/#1}}
\newcommand\BibTeX{B{\sc ib}\TeX}
\title{ALTO: Active Learning with Topic Overviews for Speeding Label Induction and Document Labeling}
\author{Forough Poursabzi-Sangdeh \\
Computer Science \\
University of Colorado\\
\smalltt{[email protected]}
\And
Jordan Boyd-Graber \\
Computer Science \\
University of Colorado\\
\smalltt{[email protected]}\protect
\AND
Leah Findlater\\
iSchool and \abr{umiacs} \\
University of Maryland\\
\smalltt{[email protected]}
\And
Kevin Seppi \\
Computer Science \\
Brigham Young University\\
\smalltt{[email protected]}
}
\begin{document}
\maketitle
\begin{abstract}
Effective text classification requires experts to annotate data with
labels; these training data are time-consuming and expensive to
obtain. If you know what labels you want, active learning can reduce
the number of labeled documents needed. However, establishing the label
set remains difficult. Annotators often lack the global knowledge
needed to induce a label set. We introduce \name{}: Active Learning
with Topic Overviews, an interactive system to help humans annotate
documents: topic models provide a global overview of what labels to
create and active learning directs them to the right documents to
label. Our user study with forty annotators shows that while active
learning by itself is best in extremely resource limited conditions,
topic models (even by themselves) lead to better label sets, and
\name{}'s combination is best overall.
\end{abstract}
\section{Introduction}
\label{sec:introduction} \input{2016_acl_doclabel/sections/introduction}
\section{\name{}: Active Learning with Topic Overviews}
\label{sec:ALTO} \input{2016_acl_doclabel/sections/alto}
\section{Study Conditions}
\label{sec:conditions} \input{2016_acl_doclabel/sections/conditions}
\section{Data and Evaluation Metrics}
\label{sec:data_metrics} \input{2016_acl_doclabel/sections/data_metrics}
\section{Synthetic Experiments}
\label{sec:synthetic_exp}\input{2016_acl_doclabel/sections/synthetic_exp}
\section{User Study}
\label{sec:user_exp_results}\input{2016_acl_doclabel/sections/user_exp_results}
\section{Related Work}
\label{sec:related_work}\input{2016_acl_doclabel/sections/related_work}
\section{Conclusion and Future Work}
\label{sec:conclusion}\input{2016_acl_doclabel/sections/conclusion}
%\clearpage
\section*{Acknowledgments}
We thank the anonymous reviewers, Philip Resnik, and Burr Settles for
their insightful comments. We also thank Nikolaos Aletras for
providing the automatic topic labeling code. Boyd-Graber and
Poursabzi-Sangdeh's contribution is supported by \abr{nsf} Grant
\abr{ncse}-1422492; Findlater, Seppi, and Boyd-Graber's contribution
is supported by collaborative \abr{nsf} Grant \abr{iis}-1409287
(\abr{umd}) and \abr{iis}-1409739 (\abr{byu}). Any opinions,
findings, results, or recommendations expressed here are of the
authors and do not necessarily reflect the view of the sponsor.
\bibliographystyle{style/acl2016}
\bibliography{bib/forough,bib/journal-full,bib/jbg}
\end{document}