Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dry up append-batched/delete-batched #115

Open
RickMoynihan opened this issue Oct 27, 2017 · 0 comments
Open

Dry up append-batched/delete-batched #115

RickMoynihan opened this issue Oct 27, 2017 · 0 comments

Comments

@RickMoynihan
Copy link
Member

RickMoynihan commented Oct 27, 2017

These two functions append-batched and delete-batched should be dried up as they're almost identical:

grafter/src/grafter/rdf.clj

Lines 139 to 158 in 66d80c3

(defn delete-batched
"Deletes a collection of statements from a repository in batches. The batch size is optional and default-batch-size
will be used if not specified."
([target quads]
(apply-batched target delete quads default-batch-size))
([target graph-or-quads triples-or-batch-size]
(if (number? triples-or-batch-size)
;;given repo, quads and batch size
(let [quads graph-or-quads
batch-size triples-or-batch-size]
(apply-batched target delete quads batch-size))
;;given repo, graph and triples
(let [graph graph-or-quads
triples triples-or-batch-size]
(apply-batched target (fn [repo batch] (delete repo graph batch)) triples default-batch-size))))
([target graph triples batch-size]
(apply-batched target (fn [repo batch] (delete repo graph batch)) triples batch-size)))

grafter/src/grafter/rdf.clj

Lines 101 to 122 in 66d80c3

(defn add-batched
"Adds a collection of statements to a repository in batches. The batch size is optional and default-batch-size
will be used if not specified. Some repository implementations cache added statements in memory until explicitly
flushed which can cause out-of-memory errors if a large number of statements are added through add. Spliting the
input sequence into batches limits the number of cached statements and therefore can reduce memory pressure."
([target triples]
(apply-batched target add triples default-batch-size))
([target graph-or-triples triples-or-batch-size]
(if (number? triples-or-batch-size)
;;given target triples and batch-size
(let [triples graph-or-triples
batch-size triples-or-batch-size]
(apply-batched target add triples batch-size))
;;given target graph and triples
(let [graph graph-or-triples
triples triples-or-batch-size]
(apply-batched target (fn [repo batch] (add repo graph batch)) triples default-batch-size))))
([target graph triples batch-size]
(apply-batched target (fn [repo batch] (add repo graph batch)) triples batch-size)))

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant