Skip to content

A Clojure high performance data processing system

License

Notifications You must be signed in to change notification settings

fern-flower-lab/tech.ml.dataset

 
 

Repository files navigation

tech.ml.dataset

TMD Logo

tech.ml.dataset (TMD) is a Clojure library for tabular data processing similar to Python's Pandas, or R's data.table. It supports pragmatic data-intensive work on the JVM by providing powerful abstractions that simplify implementing efficient solutions to real problems. Datasets shrink in memory through columnar storage and the use of primitive arrays, packed datetime types, and string tables.

Unlike in Python or R, TMD datasets are functional, which means they're easier to reason about.

Installing

Installation instructions for your favorite build system (lein, deps.edn, etc...) can be found at Clojars, where the library is hosted:

Clojars Project

Verifying Installation

user> (require 'tech.v3.dataset)
nil
user> (->> (System/getProperties)
           (map (fn [[k v]] {:k k :v (apply str (take 40 (str v)))}))
           (tech.v3.dataset/->>dataset {:dataset-name "My Truncated System Properties"}))

My Truncated System Properties [53 2]:

|                         :k |                                       :v |
|----------------------------|------------------------------------------|
|                sun.desktop |                                    gnome |
|                awt.toolkit |                     sun.awt.X11.XToolkit |
| java.specification.version |                                       11 |
|            sun.cpu.isalist |                                          |
|           sun.jnu.encoding |                                    UTF-8 |
|            java.class.path | src:resources:target/classes:/home/harol |
|             java.vm.vendor |                                   Ubuntu |
|        sun.arch.data.model |                                       64 |
|            java.vendor.url |                      https://ubuntu.com/ |
|              user.timezone |                           America/Denver |
|                        ... |                                      ... |
|                    os.arch |                                    amd64 |
| java.vm.specification.name |       Java Virtual Machine Specification |
|        java.awt.printerjob |                   sun.print.PSPrinterJob |
|         sun.os.patch.level |                                  unknown |
|          java.library.path | /usr/java/packages/lib:/usr/lib/x86_64-l |
|               java.vm.info |                      mixed mode, sharing |
|                java.vendor |                                   Ubuntu |
|            java.vm.version |      11.0.17+8-post-Ubuntu-1ubuntu222.04 |
|    sun.io.unicode.encoding |                            UnicodeLittle |
|        apple.awt.UIElement |                                     true |
|         java.class.version |                                     55.0 |

📚 Documentation 📚

The best place to start is the "Getting Started" topic in the documentation: https://techascent.github.io/tech.ml.dataset/000-getting-started.html

The "Walkthrough" topic provides long-form examples of processing real data: https://techascent.github.io/tech.ml.dataset/100-walkthrough.html

The "Quick Reference" topic summarizes many of the most frequently used functions: https://techascent.github.io/tech.ml.dataset/200-quick-reference.html

The API docs document every available function: https://techascent.github.io/tech.ml.dataset/

The provided Java API (javadoc / with frames) and sample program (source) show how to use TMD from Java.

Questions / Community


Related Projects and Notes

License

Copyright © 2023 Complements of TechAscent, LLC

Distributed under the Eclipse Public License either version 1.0 or (at your option) any later version.

About

A Clojure high performance data processing system

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Clojure 87.1%
  • Java 12.2%
  • Other 0.7%