forked from broadinstitute/fiss
-
Notifications
You must be signed in to change notification settings - Fork 0
/
changelog.txt
325 lines (245 loc) · 16.4 KB
/
changelog.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
=======================================================================
Change Log for FISSFC: the (Fi)recloud (S)ervice (S)elector
=======================================================================
Terms used below: HL = high level interface, LL = low level interface
v0.16.31 - Changed Dockerfile base image to python 3; suppress Python 2
deprecation warning from cryptography module; HL: mop updated to
find files in attributes with array and compound data structure
values, attr_delete updated to work on pair attributes, attr_get and
attr_list updated to return or elide reference data in workspace.
v0.16.30 - HL: hotfix - re-enabled making methods public.
v0.16.29 - LL: added fields parameter to get_workspace; HL: space_exists,
attr_get, attr_copy, mop, validate_file_attrs, and config_validate
updated to use fields parameter in call to get_workspace; py3 fixes:
easy_install removed from setup.py to enable compatibility with
latest setuptools, tempfile generated in fccore.edit_text now opens
in mode 'w' instead of 'w+b' to work with both py2 and py3 str.
v0.16.28 - LL: added fields parameter to list_workspaces; HL: space_list and
space_search updated to use fields parameter in call to
list_workspaces, proj_list fixed to enable py3 compatibility.
v0.16.27 - HL: added new function validate_file_attrs, which takes all
file attributes in a given workspace and entities contained within
and checks if they exist/the user has access to them; LL: Bugfix,
default root_url of API can now be set via CLI or interactively;
previously it could only be effectively modified via .fissconfig.
v0.16.26 - LL: added copyFilesWithPrefix parameter to clone_workspace,
create_submission no longer includes entity name and type in its
payload when they are not set; HL: added copyFilesWithPrefix
parameter to space_clone, config_start no longer requires entity to
be set in order to enable running non-data model configs.
v0.16.25 - HL: multiple enhancements to mop; only delete unreferenced files
generated by workflows, ignore more files generated by the execution
engine (e.g. rc, script, stdout, stderr), report file sizes, allow
user to include/exclude files via glob.
v0.16.24 - Hotfix: corrected error in api.py due to difference in user ID
location when run from a Google Cloud VM; setup.py updated to
explicitly designate long_description_content_type as text/plain.
v0.16.23 - LL: get_entities_tsv updated with ability to specify ordered
attributes and data model, upload_entities and upload_entities_tsv
updated to enable use of flexible data model API; HL: added back
entity_tsv with updated logic to detect if content is text to print
or an archive for download and support for specifying ordered
attributes and data model, entity_import updated to specify data
model; group expansion (used for giving method/config ACLs to
groups) made recursive to handle nested groups.
v0.16.22 - LL: enabled namespace, name, and snapshot ID filtering for
repository listings, new command whoami returns user id; HL: updated
listing calls to enable new filters, added capability to set method
ACLs for groups, new command set_config_acl for setting ACLs on
configs in the methods repo; google-auth updated to use current
release.
v0.16.21 - api.py will attempt to update credentials on a refresh error; typo
fixes to several HL command descriptions; setup.py now requires a
minimum version of setuptools (40.3.0); added Groups API to LL.
v0.16.20 - Updated LL command get_repository_method with documentation link and
new parameter wdl_only; new HL commands config_wdl and meth_wdl
use the updated LL command to retrieve WDL from the workspace config
and method repository respectively.
v0.16.19 - Added config_diff HL command, HL command mop updated to work in
Python 3 and fail with proper error message when workspace bucket is
inaccessible.
v0.16.18 - Added HL export commands for sample, pair and participant sets.
v0.16.17 - HL pair_list and participant_list cmds now operate on containers,
as done for sample_list in previous release; with pair_list
supporting pair, pair_set, participant and workspace; and
participant_list supporting participant, participant_set,
and workspace; extended HL tests accordingly
v0.16.16 - Added pair_list HL command; enlarged scope of applicability of
sample_list, to not only workspaces but all sample container
types: pair, sample_set, workspace, participant; added pair_list
HL function and new tests for both [pair,sample]_list; internal
HL function __get_entities now paginates by default, to enable
more calls to complete robustly; HL tests can now REUSE_SPACE for
faster execution during repeated testing
v0.16.15 - Hotfix: Fixed bug preventing gcloud tool installation.
v0.16.14 - Hotfix: Fixed warning messages that would cause errors if executed.
v0.16.13 - Dockerfile updated to use local files rather than pull from pypi;
Enhanced installation of Google Cloud SDK and ensured its
availability to subprocesses; Fixed a broken LL test.
v0.16.12 - Removed gcloud requirement from README (automatic as of v0.16.11);
Updated Dockerfile and added .dockerignore
v0.16.11 - On install, will install gcloud if not available and not running in
Google AppEngine; Now attempts to set authentication credentials if
they have not been set; Added license; Added new LL functions:
get_method_configurations() and get_api_methods_definitions();
Removed LL function get_bucket().
v0.16.10 - Formatting cleanup to HL commands config_get and config_template;
fixed tests that were broken due to change in API response code for
space_delete (200 -> 202).
v0.16.9 - Hotfix; modified how subprocess.call() is utilized to avoid
problems on Google AppEngine, where it is prohibited.
v0.16.8 - Hotfix; entity_import corrected to only strip newline characters.
v0.16.7 - Enabled LL and HL access to the linkExistingEntities field of the
copy entities API, enabling copying set entities to workspaces that
already contain some portion of the entities comprising the set.
v0.16.6 - Hotfix; workspace method config LL commands updated to comply with
API changes; setup.py updated to reflect current authorship and
contact information.
v0.16.5 - Added 3 new HL commands - config_stop(), entity_delete(), and
get_bucket() - and one new LL command - overwrite_workspace_config();
Updated 2 HL commands - space_delete(): accept new API return codes,
update_repository_method(): added snapshot comment string - and one
LL command - config_parse(): added new precedence logic and ability
to specify arbitrary attributes via **kwargs; Updated authentication
to use google.auth instead of the deprecated oauth2client, additional
related enhancements include use of a single requests session for the
duration of use (fixed a failure mode in supervisor caused by the
creation of too many connections in rapid succession) and the ability
to specify credentials file via config or CLI.
v0.16.4 - Correctly escape characters in JSON/CLI interaction; Allow Service
Account FireCloud users to run FISS; Consolidated into a global fcconfig
and cache credentials; Fixed status to health change; Enable the updating
of entities by allowing the 'update:' header; Enable access token caching;
Fix for API breaking changes in create workspace/clone_workspace.
v0.16.3 - HL flow_start() renamed to config_start, because only configs can
actually be submitted for execution (i.e. 'started'), and only
when residing in a workspace; other HL flow_* commands renamed to
meth_*, to reflect that they operate upon methods in the FC method
repo; added new config_put(), config_new() and config_template HL
commands; added more HL tests and refactored others; introduced new
main_as_cli() routine as the place which calls the true main() but
then prints the returned result to stdout [allowing existing main()
to be used in a more function-oriented way; added fccore routines
which facilitate editing content (e.g. files, method configs)
v0.16.1 - Completed print stmt refactoring started in 0.16.0; added entity_type
to config state object; added attr_list & config_delete high level
commands, and additional regression tests
v0.16.0 - Major refactoring: yapsy plugin system removed, replaced with
ConfigParser module, which employs simpler and clearer INI-style
config files (similar to GIT config); to enable high level API
(HL) to be called from other Python scripts (not just UNIX cli),
discontinue using print statements to convey results of HL calls,
in favor of returning objects (which the CLI repl will simply
print at exit; put root api_url into config state, to greatly
simplify high- and low-level API code by eliminating need for
each CMD and API func to accept a root_url param
v0.15.2 - Introduce __get wrapper for requests.get(), to lessen headers
boilerplate in the code and make it possible for --verbose flag
to be applied to ALL of the low-level API (e.g. to display the
URI of REST calls) at once, without having to pass verbosity flag
up and down call stack
v0.15.1 - New high-level 'config_get' command; adjust the low-level methods
create_workspace_config() and update_workspace_config() to reflect
newer FC api; plus contributed fixes to submission & queue routines
v0.15.0 - New 'config_validate' and 'runnable' commands for evaluating configurations
against the state of the data model in a workspace.
v0.14.1 - Bugfixes and validation for supervisor mode. Make flow_set_acl smarter,
and make the snapshot_id argument be optional, defaulting to the latest.
v0.14.0 - proj_list command for listing available billing projects. Rename
recover_supervisor to supervise_recover, for conceptual symmetry.
v0.13.2 - Fix getting workspace attributes when not using the attributes flag.
v0.13.1 - More helpful error checking for loadfile format during entity_import.
v0.13.0 - Some fixes to match API changes in FireCloud Orchestration and
several new commands:
supervise -- a robust mechanism of submitting related
workflows similar to Firehose workflows. Workflows described in DOT
can be submitted and monitored in one command, and fissfc will
monitor the entire workflow until completion.
recover_supervisor -- supervise saves workflow execution state after
each API call, so that the monitor can be restarted with this
command in case of a FISS crash.
attr_set -- set attributes in a workspace.
attr_delete -- delete attributes in a workspace.
attr_copy -- copy attributes from one workspace to another.
entity_copy -- copy entities from one workspace to another. Note that
referential constraints currently limit the ability to overwrite
entities in the target workspace.
v0.12.0 - Large refactor of argument handling for FISS. Almost all arguments
are now passed using flags instead of positional arguments, enabling
the argument namespace to be shared by command iterators. This allows
sset_loop (a new looping command) to call multiple FISS commands
without having to reparse arguments.
New FISS command for submitting workflows, 'flow_submit'.
v0.11.0 - New endpoint for paginated entity queries, also used by attr_get
to more reliably return attributes on large workspaces.
Also added new FISS command `attr_fill_null`, which will assign null
sentinel files to entities which do not have a valid attribute. This,
in a sense, "fills in" the sparse matrix of data for a given list of
attributes.
v0.10.1 - Fix Py3 compatibility issues.
v0.10.0 - Remove `fiss` entrypoint, to not overwrite fiss 1.0 installs. New fiss
"mop" command, to clean files that are not referenced in a workspace
attribute from the underlying google bucket.
v0.9.6 - Fix expected response code for entity_import
v0.9.5 - Fix urllib compatibility issue in upload_entities
v0.9.4 - Add ability to change inputs and other configuration details in a
workspace configuration
v0.9.3 - Improved documentation. Fixed url for method configurations in a workspace.
v0.9.2 - Remove file-based check for getting application default credentials,
improving support for Jupyter-based sessions.
v0.9.1 - Fix expected return code for flow_new.
v0.9.0 - Replace Httplib2 with Requests, which resolves the SSL handshake issue
for python3. Added six to required packages, making FireCloud fully
compatible with python 2 & 3. Add links to FireCloud's swagger
documentation.
v0.8.1 - Improved test coverage. Enhanced interactive classes. Fixed error logic
to match new server response.
v0.8.0 - firecloud.api now boasts 100% coverage of existing (useful) API endpoints.
Several api endpoints for manipulating NIH credentials and profile info
will deliberately remain unimplemented unless a good reason arises. Any
discrepancies are noted in the api module's docstring; see
help(firecloud.api) for a lengthy overview.
New Method and Configuration objects.
>>> m = Method("broad-firecloud-gdac", "gistic2", 1)
v0.7.4 - FISS functions can now display their source code with the -F option.
v0.7.3 - Improved documentation, with docstrings for every public function.
This enables python's builtin help system to be used from the REPL.
To see all the available firecloud api calls:
% python
>>> import firecloud
>>> help(firecloud.api)
v0.7.2 - Fix some commands that broke with the API changes
v0.7.1 - Add workspace functions for importing entities and creating entity sets.
v0.7.0 - Renamed package "firecloud", replacing "fissfc"
- firecloud_cli.py renamed fiss.py
- firecloud_api.py renamed api.py
Added workspace class for more flexible python interactivity. Example:
>>> w = Workspace.new("broad-firecloud-testing", "my_workspace")
>>> w.bucket()
"fc-92e4ic0a-60fb-454a-8ue0-d0efea803421"
>>> w.lock()
Added better error messaging for unexpected FireCloud Errors
v0.6.2 - Add '--protected' option for workspace creation.
v0.6.1 - Firecloud production API is now https://api.firecloud.org/api
v0.6.0 - Refactored the argument structure for inferring a default project.
Instead of passing the namespace as an optional input to a
subcommand, the project can be overriden explicitly with the -p
top-level flag. This cleans up the syntax for many commands, and
makes the help clearer.
Add --all option to space_list
attr_get:
* attributes now given a a list of positional arguments,
instead of comma-separated
* entity type limited to firecloud supported entities
Fixed help messages for many commands
v0.5.0 - Add python 3 support, so that fissfc should be installable to
either version. A TLS issue may still prevent fissfc from
working on some versions of python.
v0.4.1 - Add attr_get subcommand, as a replacement for fiss annot_get.
Clarify error message.
v0.4.0 - Implement default value for google project into most subcommands
v0.3.2 - Add check in firecloud_api for the existence of gcloud credentials
v0.3.1 - Add confirmation prompt for delete commands, and a
Flag to override them.
v0.3.0 - Added support for Yapsy plugins.
v0.0.1 - Initial Release