forked from arangodb/arangodb
-
Notifications
You must be signed in to change notification settings - Fork 1
/
CHANGELOG
10782 lines (7073 loc) · 413 KB
/
CHANGELOG
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
devel
-----
* changed the thread handling in the scheduler. `--server.threads` will be
the maximum number of threads for the scheduler.
The hidden option `--server.maximal-threads` is now obsolete.
The hidden options `--server.queue-size`, `--server.prioN-size` (N = 1 or 2)
control the size of the internal queues. The defaults are sensible, do not
change these settings.
* use sparse indexes in more cases now, when it is clear that the index attribute
value cannot be null
* Added load balancer support and user-restriction to cursor API.
If a cursor is accessed on a different coordinator than where it was created,
the requests will be forwarded to the correct coordinator. If a cursor is
accessed by a different user than the one who created it, the request will
be denied.
* if authentication is turned on requests to databases by users with insufficient rights
will be answered with the HTTP forbidden (401) response.
* upgraded bundled RocksDB library version to 5.14
* added key generators `uuid` and `padded`
The `uuid` key generator generates universally unique 128 bit keys, which are
stored in hexadecimal human-readable format.
The `padded` key generator generates keys of a fixed length (16 bytes) in
ascending lexicographical sort order.
* The REST API of `/_admin/status` changed: Field "mode" is now named "operationMode",
field "writeOpsEnabled" no longer exists and is replaced by field "readOnly" with
the inverted meaning.
* added `COUNT_DISTINCT` AQL function
* make AQL optimizer rule `collect-in-cluster` optimize aggregation functions
`AVERAGE`, `VARIANCE`, `STDDEV`, `UNIQUE`, `SORTED_UNIQUE` and `COUNT_DISTINCT`
in a cluster by pushing parts of the aggregation onto the DB servers and only
doing the total aggregation on the coordinator
* replace JavaScript functions FULLTEXT, NEAR, WITHIN and WITHIN_RECTANGLE with
regular AQL subqueries via a new optimizer rule "replace-function-with-index".
* the existing "fulltext-index-optimizer" optimizer rule has been removed because its
duty is now handled by the "replace-function-with-index" rule.
* The `NEAR` AQL function now does not default to a limit of 100 documents any more.
* added option "--latency true" option to arangoimport. Lists microsecond latency
statistics on 10 second intervals.
* fixed internal issue #2256: ui, document id not showing up when deleting a document
* fixed internal issue #2163: wrong labels within foxx validation of service
input parameters
* fixed internal issue #2160: fixed misplaced tooltips in indices view
* Added exclusive option for rocksdb collections. Modifying AQL queries can
now set the exclusive option as well as it can be set on JavaScript transactions.
* added optimizer rule "optimize-subqueries", which makes qualifying subqueries
return less data
The rule fires in the following situations:
* in case only a few results are used from a non-modifying subquery, the rule
will add a LIMIT statement into the subquery. For example
LET docs = (
FOR doc IN collection
FILTER ...
RETURN doc
)
RETURN docs[0]
will be turned into
LET docs = (
FOR doc IN collection
FILTER ...
LIMIT 1
RETURN doc
)
RETURN docs[0]
Another optimization performed by this rule is to modify the result value
of subqueries in case only the number of results is checked later. For example
RETURN LENGTH(
FOR doc IN collection
FILTER ...
RETURN doc
)
will be turned into
RETURN LENGTH(
FOR doc IN collection
FILTER ...
RETURN true
)
This saves copying the document data from the subquery to the outer scope and may
enable follow-up optimizations.
* fixed issue #5035: fixed a vulnerability issue within the web ui's index view
* fixed Foxx queues bug when queues are created in a request handler with an
ArangoDB authentication header
* abort startup when using SSLv2 for a server endpoint, or when connecting with
a client tool via an SSLv2 connection.
SSLv2 has been disabled in the OpenSSL library by default in recent versions
because of security vulnerabilities inherent in this protocol.
As it is not safe at all to use this protocol, the support for it has also
been stopped in ArangoDB. End users that use SSLv2 for connecting to ArangoDB
should change the protocol from SSLv2 to TLSv12 if possible, by adjusting
the value of the `--ssl.protocol` startup option.
* added `overwrite` option to the `document rest-handler` to allow for easier syncing.
This implements almost the much inquired UPSERT. In reality it is a REPSERT
(replace/insert) because only replacement and not modification of documents
is possible. The option does not work in cluster collections with custom
sharding.
* added startup option `--log.escape`
This option toggles the escaping of log output.
If set to `true` (which is the default value), then the logging will work
as before, and the following characters in the log output are escaped:
* the carriage return character (hex 0d)
* the newline character (hex 0a)
* the tabstop character (hex 09)
* any other characters with an ordinal value less than hex 20
If the option is set to `false`, no characters are escaped. Characters with
an ordinal value less than hex 20 will not be printed in this mode but will
be replaced with a space character (hex 20).
A side effect of turning off the escaping is that it will reduce the CPU
overhead for the logging. However, this will only be noticable when logging
is set to a very verbose level (e.g. debug or trace).
* increased the default values for the startup options `--javascript.gc-interval`
from every 1000 to every 2000 requests, and for `--javascript.gc-frequency` from
30 to 60 seconds
This will make the V8 garbage collection run less often by default than in previous
versions, reducing CPU load a bit and leaving more contexts available on average.
* added `/_admin/repair/distributeShardsLike` that repairs collections with
distributeShardsLike where the shards aren't actually distributed like in the
prototype collection, as could happen due to internal issue #1770
* Fixed issue #4271: Change the behavior of the `fullCount` option for AQL query
cursors so that it will only take into account `LIMIT` statements on the top level
of the query.
`LIMIT` statements in subqueries will not have any effect on the `fullCount` results
any more.
* We added a new geo-spatial index implementation. On the RocksDB storage engine all
installations will need to be upgraded with `--database.auto-upgrade true`. New geo
indexes will now only report with the type `geo` instead of `geo1` or `geo2`.
The index types `geo1` and `geo2` are now deprecated.
Additionally we removed the deprecated flags `constraint` and `ignoreNull` from geo
index definitions, these fields were initially deprecated in ArangoDB 2.5
* Add revision id to RocksDB values in primary indexes to speed up replication (~10x).
* PR #5238: Create a default pacing algorithm for arangoimport to avoid TimeoutErrors
on VMs with limited disk throughput
* Starting a cluster with coordinators and DB servers using different storage engines
is unsupported. Doing it anyway will now produce a warning on startup
* fixed issue #4919: C++ implementation of LIKE function now matches the old and correct
behaviour of the javascript implementation.
* added `--json` option to arangovpack, allowing to treat its input as plain JSON data
make arangovpack work without any configuration file
* added experimental arangodb startup option `--javascript.enabled` to enable/disable the
initialization of the V8 JavaScript engine. Only expected to work on single-servers and
agency deployments
* pull request #5201: eliminate race scenario where handlePlanChange could run infinite times
after an execution exceeded 7.4 second time span
* UI: fixed an unreasonable event bug within the modal view engine
* pull request #5114: detect shutdown more quickly on heartbeat thread of coordinator and
DB servers
* fixed issue #3811: gharial api is now checking existence of `_from` and `_to` vertices
during edge creation
* There is new method `_profileQuery` on the database object to execute a query and
print an explain with annotated runtime information.
* Query cursors can now be created with option `profile`, with a value of 0, 1 or 2.
This will cause queries to include more statistics in their results and will allow tracing
of queries.
* fixed internal issue #2147: fixed database filter in UI
* fixed internal issue #2149: number of documents in the UI is not adjusted after moving them
* fixed internal issue #2150: UI - loading a saved query does not update the list of bind
parameters
* removed option `--cluster.my-local-info` in favor of persisted server UUIDs
The option `--cluster.my-local-info` was deprecated since ArangoDB 3.3.
* added new collection property `cacheEnabled` which enables in-memory caching for
documents and primary index entries. Available only when using RocksDB
* arangodump now supports `--threads` option to dump collections in parallel
* arangorestore now supports `--threads` option to restore collections in parallel
* Improvement: The AQL query planner in cluster is now a bit more clever and
can prepare AQL queries with less network overhead.
This should speed up simple queries in cluster mode, on complex queries it
will most likely not show any performance effect.
It will especially show effects on collections with a very high amount of Shards.
* removed remainders of dysfunctional `/_admin/cluster-test` and `/_admin/clusterCheckPort`
API endpoints and removed them from documentation
* remove `_admin/echo` handler
* added new query option `stream` to enable streaming query execution via the
`POST /_api/cursor` rest interface.
* fixed issue #4698: databases within the UI are now displayed in a sorted order.
* Behavior of permissions for databases and collections changed:
The new fallback rule for databases for which an access level is not explicitly specified:
Choose the higher access level of:
* A wildcard database grant
* A database grant on the `_system` database
The new fallback rule for collections for which an access level is not explicitly specified:
Choose the higher access level of:
* Any wildcard access grant in the same database, or on "*/*"
* The access level for the current database
* The access level for the `_system` database
* fixed issue #4583: add AQL ASSERT and AQL WARN
* renamed startup option `--replication.automatic-failover` to
`--replication.active-failover`
using the old option name will still work in ArangoDB 3.4, but the old option
will be removed afterwards
* index selectivity estimates for RocksDB engine are now eventually consistent
This change addresses a previous issue where some index updates could be
"lost" from the view of the internal selectivity estimate, leading to
inaccurate estimates. The issue is solved now, but there can be up to a second
or so delay before updates are reflected in the estimates.
* support `returnOld` and `returnNew` attributes for in the following HTTP REST
APIs:
* /_api/gharial/<graph>/vertex/<collection>
* /_api/gharial/<graph>/edge/<collection>
The exception from this is that the HTTP DELETE verb for these APIs does not
support `returnOld` because that would make the existing API incompatible
* fixed internal issue #478: remove unused and undocumented REST API endpoints
_admin/statistics/short and _admin/statistics/long
These APIs were available in ArangoDB's REST API, but have not been called by
ArangoDB itself nor have they been part of the documented API. They have been
superseded by other REST APIs and were partially dysfunctional. Therefore
these two endpoints have been removed entirely.
* fixed issue #1532: reload users on restore
* fixed internal issue #1475: when restoring a cluster dump to a single server
ignore indexes of type primary and edge since we mustn't create them here.
* fixed internal issue #1439: improve performance of any-iterator for RocksDB
* issue #1190: added option `--create-database` for arangoimport
* UI: updated dygraph js library to version 2.1.0
* renamed arangoimp to arangoimport for consistency
Release packages will still install arangoimp as a symlink so user scripts
invoking arangoimp do not need to be changed
* UI: Shard distribution view now has an accordion view instead of displaying
all shards of all collections at once.
* fixed issue #4393: broken handling of unix domain sockets in JS_Download
* added AQL function `IS_KEY`
this function checks if the value passed to it can be used as a document key,
i.e. as the value of the `_key` attribute
* added AQL functions `SORTED` and `SORTED_UNIQUE`
`SORTED` will return a sorted version of the input array using AQL's internal
comparison order
`SORTED_UNIQUE` will do the same, but additionally removes duplicates.
* added C++ implementation for AQL functions `DATE_NOW`, `DATE_ISO8601`,
`DATE_TIMESTAMP`, `IS_DATESTRING`, `DATE_DAYOFWEEK`, `DATE_YEAR`,
`DATE_MONTH`, `DATE_DAY`, `DATE_HOUR`, `DATE_MINUTE`, `DATE_SECOND`,
`DATE_MILLISECOND`, `DATE_DAYOFYEAR`, `DATE_ISOWEEK`, `DATE_LEAPYEAR`,
`DATE_QUARTER`, `DATE_DAYS_IN_MONTH`, `DATE_ADD`, `DATE_SUBTRACT`,
`DATE_DIFF`, `DATE_COMPARE`, `TRANSLATE` and `SHA512`
* fixed a bug where clusterinfo missed changes to plan after agency
callback is registred for create collection
* Foxx manifest.json files can now contain a $schema key with the value
of "http://json.schemastore.org/foxx-manifest" to improve tooling support.
* fixed agency restart from compaction without data
v3.3.8 (XXXX-XX-XX)
-------------------
* included version of ArangoDB Starter (`arangodb` binary) updated to v0.10.11,
see [Starter changelog](https://github.com/arangodb-helper/arangodb/blob/master/CHANGELOG.md)
* added arangod startup option `--dump-options` to print all configuration parameters
as a JSON object
* fixed: (Enterprise only) If you restore a SmartGraph where the collections
are still existing and are supposed to be dropped on restore we ended up in
duplicate name error. This is now gone and the SmartGraph is correctly restored.
* fix lookups by `_id` in smart graph edge collections
* improve startup resilience in case there are datafile errors (MMFiles)
also allow repairing broken VERSION files automatically on startup by
specifying the option `--database.ignore-datafile-errors true`
* fix issue #4582: UI query editor now supports usage of empty string as bind parameter value
* fixed internal issue #2148: Number of documents found by filter is misleading in web UI
* added startup option `--database.required-directory-state`
using this option it is possible to require the database directory to be
in a specific state on startup. the options for this value are:
- non-existing: database directory must not exist
- existing: database directory must exist
- empty: database directory must exist but be empty
- populated: database directory must exist and contain specific files already
- any: any state allowed
* field "$schema" in Foxx manifest.json files no longer produce warnings
* added `@arangodb/locals` module to expose the Foxx service context as an
alternative to using `module.context` directly.
* `db._executeTransaction` now accepts collection objects as collections.
* supervision can be put into maintenance mode
v3.3.7 (2018-04-11)
-------------------
* added hidden option `--query.registry-ttl` to control the lifetime of cluster AQL
query parts
* fixed internal issue #2237: AQL queries on collections with replicationFactor:
"satellite" crashed arangod in single server mode
* fixed restore of satellite collections: replicationFactor was set to 1 during
restore
* fixed dump and restore of smart graphs:
a) The dump will not include the hidden shadow collections anymore, they were dumped
accidentially and only contain duplicated data.
b) Restore will now ignore hidden shadow collections as all data is contained
in the smart-edge collection. You can manually include these collections from an
old dump (3.3.5 or earlier) by using `--force`.
c) Restore of a smart-graph will now create smart collections properly instead
of getting into `TIMEOUT_IN_CLUSTER_OPERATION`
* fixed issue in AQL query optimizer rule "restrict-to-single-shard", which
may have sent documents to a wrong shard in AQL INSERT queries that specified
the value for `_key` using an expression (and not a constant value)
Important: if you were affected by this bug in v3.3.5 it is required that you
recreate your dataset in v3.3.6 (i.e. dumping and restoring) instead of doing
a simple binary upgrade
* added /_admin/status HTTP API for debugging purposes
* added ArangoShell helper function for packaging all information about an
AQL query so it can be run and analyzed elsewhere:
query = "FOR doc IN mycollection FILTER doc.value > 42 RETURN doc";
require("@arangodb/aql/explainer").debugDump("/tmp/query-debug-info", query);
Entitled users can send the generated file to the ArangoDB support to facilitate
reproduction and debugging.
* added hidden option `--server.ask-jwt-secret`. This is an internal option
for debugging and should not be exposed to end-users.
* fix for internal issue #2215. supervision will now wait for agent to
fully prepare before adding 10 second grace period after leadership change
* fixed internal issue #2215's FailedLeader timeout bug
v3.3.5 (2018-03-28)
-------------------
* fixed issue #4934: Wrong used GeoIndex depending on FILTER order
* make build id appear in startup log message alongside with other version info
* make AQL data modification operations that are sent to all shards and that are
supposed to return values (i.e. `RETURN OLD` or `RETURN NEW`) not return fake
empty result rows if the document to be updated/replaced/removed was not present
on the target shard
* added AQL optimizer rule `restrict-to-single-shard`
This rule will kick in if a collection operation (index lookup or data
modification operation) will only affect a single shard, and the operation can be
restricted to the single shard and is not applied for all shards. This optimization
can be applied for queries that access a collection only once in the query, and that
do not use traversals, shortest path queries and that do not access collection data
dynamically using the `DOCUMENT`, `FULLTEXT`, `NEAR` or `WITHIN` AQL functions.
Additionally, the optimizer will only pull off this optimization if can safely
determine the values of all the collection's shard keys from the query, and when the
shard keys are covered by a single index (this is always true if the shard key is
the default `_key`)
* display missing attributes of GatherNodes in AQL explain output
* make AQL optimizer rule `undistribute-remove-after-enum-coll` fire in a few
more cases in which it is possible
* slightly improve index selection for the RocksDB engine when there are multiple
competing indexes with the same attribute prefixes, but different amount of
attributes covered. In this case, the more specialized index will be preferred
now
* fix issue #4924: removeFollower now prefers to remove the last follower(s)
* added "collect-in-cluster" optimizer rule to have COLLECT WITH COUNT queries
without grouping being executed on the DB servers and the coordinator only summing
up the counts from the individual shards
* fixed issue #4900: Nested FOR query uses index but ignores other filters
* properly exit v8::Context in one place where it was missing before
* added hidden option `--cluster.index-create-timeout` for controlling the
default value of the index creation timeout in cluster
under normal circumstances, this option does not need to be adjusted
* increase default timeout for index creation in cluster to 3600s
* fixed issue #4843: Query-Result has more Docs than the Collection itself
* fixed the behavior of ClusterInfo when waiting for current to catch
up with plan in create collection.
* fixed issue #4827: COLLECT on edge _to field doesn't group distinct values as expected (MMFiles)
v3.3.4 (2018-03-01)
-------------------
* fix AQL `fullCount` result value in some cluster cases when it was off a bit
* fix issue #4651: Simple query taking forever until a request timeout error
* fix issue #4657: fixed incomplete content type header
* Vastly improved the Foxx Store UI
* fix issue #4677: AQL WITH with bind parameters results in "access after data-modification"
for two independent UPSERTs
* remove unused startup option `--ldap.permissions-attribute-name`
* fix issue #4457: create /var/tmp/arangod with correct user in supervisor mode
* remove long disfunctional admin/long_echo handler
* fixed Foxx API:
* PUT /_api/foxx/service: Respect force flag
* PATCH /_api/foxx/service: Check whether a service under given mount exists
* internal issue #1726: supervision failed to remove multiple servers
from health monitoring at once.
* more information from inception, why agent is activated
* fixed a bug where supervision tried to deal with shards of virtual collections
* fix internal issue #1770: collection creation using distributeShardsLike yields
errors and did not distribute shards correctly in the following cases:
1. If numberOfShards * replicationFactor % nrDBServers != 0
(shards * replication is not divisible by DBServers).
2. If there was failover / move shard case on the leading collection
and creating the follower collection afterwards.
* fix timeout issues in replication client expiration
* added missing edge filter to neighbors-only traversals
in case a filter condition was moved into the traverser and the traversal was
executed in breadth-first mode and was returning each visited vertex exactly
once, and there was a filter on the edges of the path and the resulting vertices
and edges were not used later, the edge filter was not applied
* fixed issue #4160: Run arangod with "--database.auto-upgrade" option always crash silently without error log
* fix internal issue #1848: AQL optimizer was trying to resolve attribute accesses
to attributes of constant object values at query compile time, but only did so far
the very first attribute in each object
this fixes https://stackoverflow.com/questions/48648737/beginner-bug-in-for-loops-from-objects
* fix inconvenience: If we want to start server with a non-existing
--javascript.app-path it will now be created (if possible)
* fixed: REST API `POST _api/foxx` now returns HTTP code 201 on success, as documented.
returned 200 before.
* fixed: REST API `PATCH _api/foxx/dependencies` now updates the existing dependencies
instead of replacing them.
* fixed: Foxx upload of single javascript file. You now can upload via http-url pointing
to a javascript file.
* fixed issue #4395: If your foxx app includes an `APP` folder it got
accidently removed by selfhealing this is not the case anymore.
* fixed internal issue #1969 - command apt-get purge/remove arangodb3e was failing
v3.3.3 (2018-01-16)
-------------------
* fix issue #4272: VERSION file keeps disappearing
* fix internal issue #81: quotation marks disappeared when switching table/json
editor in the query editor ui
* added option `--rocksdb.throttle` to control whether write-throttling is enabled
Write-throttling is turned on by default, to reduce chances of compactions getting
too far behind and blocking incoming writes.
* fixed issue #4308: Crash when getter for error.name throws an error (on Windows)
* UI: fixed a query editor caching and parsing issue
* Fixed internal issue #1683: fixes an UI issue where a collection name gets wrongly cached
within the documents overview of a collection.
* Fixed an issue with the index estimates in RocksDB in the case a transaction is aborted.
Former the index estimates were modified if the transaction commited or not.
Now they will only be modified if the transaction commited successfully.
* UI: optimized login view for very small screen sizes
* Truncate in RocksDB will now do intermediate commits every 10.000 documents
if truncate fails or the server crashes during this operation all deletes
that have been commited so far are persisted.
* make the default value of `--rocksdb.block-cache-shard-bits` use the RocksDB
default value. This will mostly mean the default number block cache shard
bits is lower than before, allowing each shard to store more data and cause
less evictions from block cache
* issue #4222: Permission error preventing AQL query import / export on webui
* UI: optimized error messages for invalid query bind parameter
* UI: upgraded swagger ui to version 3.9.0
* issue #3504: added option `--force-same-database` for arangorestore
with this option set to true, it is possible to make any arangorestore attempt
fail if the specified target database does not match the database name
specified in the source dump's "dump.json" file. it can thus be used to
prevent restoring data into the "wrong" database
The option is set to `false` by default to ensure backwards-compatibility
* make the default value of `--rocksdb.block-cache-shard-bits` use the RocksDB
default value. This will mostly mean the default number block cache shard
bits is lower than before, allowing each shard to store more data and cause
less evictions from block cache
* fixed issue #4255: AQL SORT consuming too much memory
* fixed incorrect persistence of RAFT vote and term
v3.3.2 (2018-01-04)
-------------------
* fixed issue #4199: Internal failure: JavaScript exception in file 'arangosh.js'
at 98,7: ArangoError 4: Expecting type String
* fixed issue in agency supervision with a good server being left in
failedServers
* distinguish isReady and allInSync in clusterInventory
* fixed issue #4197: AQL statement not working in 3.3.1 when upgraded from 3.2.10
* do not reuse collection ids when restoring collections from a dump, but assign new collection ids, this should prevent collection id conflicts
v3.3.1 (2017-12-28)
-------------------
* UI: displayed wrong wfs property for a collection when using RocksDB as
storage engine
* added `--ignore-missing` option to arangoimp
this option allows importing lines with less fields than specified in the CSV
header line
* changed misleading error message from "no leader" to "not a leader"
* optimize usage of AQL FULLTEXT index function to a FOR loop with index
usage in some cases
When the optimization is applied, this especially speeds up fulltext index
queries in the cluster
* UI: improved the behavior during collection creation in a cluster environment
* Agency lockup fixes for very small machines.
* Agency performance improvement by finer grained locking.
* Use steady_clock in agency whereever possible.
* Agency prevent Supervision thread crash.
* Fix agency integer overflow in timeout calculation.
v3.3.0 (2012-12-14)
-------------------
* release version
* added a missing try/catch block in the supervision thread
v3.3.rc8 (2017-12-12)
---------------------
* UI: fixed broken foxx configuration keys. Some valid configuration values
could not be edited via the ui.
* UI: pressing the return key inside a select2 box no longer triggers the modal's
success function
* UI: coordinators and db servers are now in sorted order (ascending)
v3.3.rc7 (2017-12-07)
---------------------
* fixed issue #3741: fix terminal color output in Windows
* UI: fixed issue #3822: disabled name input field for system collections
* fixed issue #3640: limit in subquery
* fixed issue #3745: Invalid result when using OLD object with array attribute in UPSERT statement
* UI: edge collections were wrongly added to from and to vertices select box during graph creation
* UI: added not found views for documents and collections
* UI: using default user database api during database creation now
* UI: the graph viewer backend now picks one random start vertex of the
first 1000 documents instead of calling any(). The implementation of
"any" is known to scale bad on huge collections with RocksDB.
* UI: fixed disappearing of the navigation label in some case special case
* UI: the graph viewer now displays updated label values correctly.
Additionally the included node/edge editor now closes automatically
after a successful node/edge update.
* fixed issue #3917: traversals with high maximal depth take extremely long
in planning phase.
v3.3.rc4 (2017-11-28)
---------------------
* minor bug-fixes
v3.3.rc3 (2017-11-24)
---------------------
* bug-fixes
v3.3.rc2 (2017-11-22)
---------------------
* UI: document/edge editor now remembering their modes (e.g. code or tree)
* UI: optimized error messages for invalid graph definitions. Also fixed a
graph renderer cleanup error.
* UI: added a delay within the graph viewer while changing the colors of the
graph. Necessary due different browser behaviour.
* added options `--encryption.keyfile` and `--encryption.key-generator` to arangodump
and arangorestore
* UI: the graph viewer now displays updated label values correctly.
Additionally the included node/edge editor now closes automatically
after a successful node/edge update.
* removed `--recycle-ids` option for arangorestore
using that option could have led to problems on the restore, with potential
id conflicts between the originating server (the source dump server) and the
target server (the restore server)
v3.3.rc1 (2017-11-17)
---------------------
* add readonly mode REST API
* allow compilation of ArangoDB source code with g++ 7
* upgrade minimum required g++ compiler version to g++ 5.4
That means ArangoDB source code will not compile with g++ 4.x or g++ < 5.4 anymore.
* AQL: during a traversal if a vertex is not found. It will not print an ERROR to the log and continue
with a NULL value, but will register a warning at the query and continue with a NULL value.
The situation is not desired as an ERROR as ArangoDB can store edges pointing to non-existing
vertex which is perfectly valid, but it may be a n issue on the data model, so users
can directly see it on the query now and do not "by accident" have to check the LOG output.
* introduce `enforceReplicationFactor` attribute for creating collections:
this optional parameter controls if the coordinator should bail out during collection
creation if there are not enough DBServers available for the desired `replicationFactor`.
* fixed issue #3516: Show execution time in arangosh
this change adds more dynamic prompt components for arangosh
The following components are now available for dynamic prompts,
settable via the `--console.prompt` option in arangosh:
- '%t': current time as timestamp
- '%a': elpased time since ArangoShell start in seconds
- '%p': duration of last command in seconds
- '%d': name of current database
- '%e': current endpoint
- '%E': current endpoint without protocol
- '%u': current user
The time a command takes can be displayed easily by starting arangosh with `--console.prompt "%p> "`.
* make the ArangoShell refill its collection cache when a yet-unknown collection
is first accessed. This fixes the following problem:
arangosh1> db._collections(); // shell1 lists all collections
arangosh2> db._create("test"); // shell2 now creates a new collection 'test'
arangosh1> db.test.insert({}); // shell1 is not aware of the collection created
// in shell2, so the insert will fail
* make AQL `DISTINCT` not change the order of the results it is applied on
* incremental transfer of initial collection data now can handle partial
responses for a chunk, allowing the leader/master to send smaller chunks
(in terms of HTTP response size) and limit memory usage
this optimization is only active if client applications send the "offset" parameter
in their requests to PUT `/_api/replication/keys/<id>?type=docs`
* initial creation of shards for cluster collections is now faster with
`replicationFactor` values bigger than 1. this is achieved by an optimization
for the case when the collection on the leader is still empty
* potential fix for issue #3517: several "filesystem full" errors in logs
while there's a lot of disk space
* added C++ implementations for AQL function `SUBSTRING()`, `LEFT()`, `RIGHT()` and `TRIM()`
* show C++ function name of call site in ArangoDB log output
this requires option `--log.line-number` to be set to *true*
* UI: added word wrapping to query editor
* UI: fixed wrong user attribute name validation, issue #3228
* make AQL return a proper error message in case of a unique key constraint
violation. previously it only returned the generic "unique constraint violated"
error message but omitted the details about which index caused the problem.
This addresses https://stackoverflow.com/questions/46427126/arangodb-3-2-unique-constraint-violation-id-or-key
* added option `--server.local-authentication`
* UI: added user roles
* added config option `--log.color` to toggle colorful logging to terminal
* added config option `--log.thread-name` to additionally log thread names
* usernames must not start with `:role:`, added new options:
--server.authentication-timeout
--ldap.roles-attribute-name
--ldap.roles-transformation
--ldap.roles-search
--ldap.superuser-role
--ldap.roles-include
--ldap.roles-exclude
* performance improvements for full collection scans and a few other operations
in MMFiles engine
* added `--rocksdb.encryption-key-generator` for enterprise
* removed `--compat28` parameter from arangodump and replication API
older ArangoDB versions will no longer be supported by these tools.
* increase the recommended value for `/proc/sys/vm/max_map_count` to a value
eight times as high as the previous recommended value. Increasing the
values helps to prevent an ArangoDB server from running out of memory mappings.
The raised minimum recommended value may lead to ArangoDB showing some startup
warnings as follows:
WARNING {memory} maximum number of memory mappings per process is 65530, which seems too low. it is recommended to set it to at least 512000
WARNING {memory} execute 'sudo sysctl -w "vm.max_map_count=512000"'
* Foxx now warns about malformed configuration/dependency names and aliases in the manifest.
v3.2.7 (2017-11-13)
-------------------
* Cluster customers, which have upgraded from 3.1 to 3.2 need to upgrade
to 3.2.7. The cluster supervision is otherwise not operational.
* Fixed issue #3597: AQL with path filters returns unexpected results
In some cases breadth first search in combination with vertex filters
yields wrong result, the filter was not applied correctly.
* fixed some undefined behavior in some internal value caches for AQL GatherNodes
and SortNodes, which could have led to sorted results being effectively not
correctly sorted.
* make the replication applier for the RocksDB engine start automatically after a
restart of the server if the applier was configured with its `autoStart` property
set to `true`. previously the replication appliers were only automatically restarted
at server start for the MMFiles engine.
* fixed arangodump batch size adaptivity in cluster mode and upped default batch size
for arangodump
these changes speed up arangodump in cluster context
* smart graphs now return a proper inventory in response to replication inventory
requests
* fixed issue #3618: Inconsistent behavior of OR statement with object bind parameters
* only users with read/write rights on the "_system" database can now execute
"_admin/shutdown" as well as modify properties of the write-ahead log (WAL)
* increase default maximum number of V8 contexts to at least 16 if not explicitly
configured otherwise.
the procedure for determining the actual maximum value of V8 contexts is unchanged
apart from the value `16` and works as follows:
- if explicitly set, the value of the configuration option `--javascript.v8-contexts`
is used as the maximum number of V8 contexts
- when the option is not set, the maximum number of V8 contexts is determined
by the configuration option `--server.threads` if that option is set. if
`--server.threads` is not set, then the maximum number of V8 contexts is the
server's reported hardware concurrency (number of processors visible
to the arangod process). if that would result in a maximum value of less than 16
in any of these two cases, then the maximum value will be increased to 16.
* fixed issue #3447: ArangoError 1202: AQL: NotFound: (while executing) when
updating collection
* potential fix for issue #3581: Unexpected "rocksdb unique constraint
violated" with unique hash index
* fixed geo index optimizer rule for geo indexes with a single (array of coordinates)
attribute.
* improved the speed of the shards overview in cluster (API endpoint /_api/cluster/shardDistribution API)
It is now guaranteed to return after ~2 seconds even if the entire cluster is unresponsive.
* fix agency precondition check for complex objects
this fixes issues with several CAS operations in the agency
* several fixes for agency restart and shutdown
* the cluster-internal representation of planned collection objects is now more
lightweight than before, using less memory and not allocating any cache for indexes
etc.
* fixed issue #3403: How to kill long running AQL queries with the browser console's
AQL (display issue)
* fixed issue #3549: server reading ENGINE config file fails on common standard
newline character
* UI: fixed error notifications for collection modifications
* several improvements for the truncate operation on collections:
* the timeout for the truncate operation was increased in cluster mode in
order to prevent too frequent "could not truncate collection" errors
* after a truncate operation, collections in MMFiles still used disk space.
to reclaim disk space used by truncated collection, the truncate actions
in the web interface and from the ArangoShell now issue an extra WAL flush
command (in cluster mode, this command is also propagated to all servers).
the WAL flush allows all servers to write out any pending operations into the
datafiles of the truncated collection. afterwards, a final journal rotate
command is sent, which enables the compaction to entirely remove all datafiles
and journals for the truncated collection, so that all disk space can be
reclaimed
* for MMFiles a special method will be called after a truncate operation so that
all indexes of the collection can free most of their memory. previously some
indexes (hash and skiplist indexes) partially kept already allocated memory
in order to avoid future memory allocations
* after a truncate operation in the RocksDB engine, an additional compaction
will be triggered for the truncated collection. this compaction removes all
deletions from the key space so that follow-up scans over the collection's key
range do not have to filter out lots of already-removed values
These changes make truncate operations potentially more time-consuming than before,
but allow for memory/disk space savings afterwards.
* enable JEMalloc background threads for purging and returning unused memory
back to the operating system (Linux only)
JEMalloc will create its background threads on demand. The number of background
threads is capped by the number of CPUs or active arenas. The background threads run
periodically and purge unused memory pages, allowing memory to be returned to the
operating system.
This change will make the arangod process create several additional threads.
It is accompanied by an increased `TasksMax` value in the systemd service configuration
file for the arangodb3 service.
* upgraded bundled V8 engine to bugfix version v5.7.492.77
this upgrade fixes a memory leak in upstream V8 described in
https://bugs.chromium.org/p/v8/issues/detail?id=5945 that will result in memory
chunks only getting uncommitted but not unmapped
v3.2.6 (2017-10-26)
-------------------
* UI: fixed event cleanup in cluster shards view
* UI: reduced cluster dashboard api calls
* fixed a permission problem that prevented collection contents to be displayed
in the web interface
* removed posix_fadvise call from RocksDB's PosixSequentialFile::Read(). This is
consistent with Facebook PR 2573 (#3505)