Hiyori CLI

Hiyori CLI is being developed to create and maintain HiyoriDB — a relational Anime and Manga Database.

Warning

This project is a work-in-progress and is not production ready.

Process

Hiyori CLI has two main primary functions:

1. Ingesting transformed data based on the Hiyori Schema ingested from available sources
2. Combining the ingested data into HiyoriDB

Hiyori also collects relational mappings from other sources (if available) and uses that to populate references in the schema. Supported references have parsers and are able to extract the ID of the entry for that source and store them in reference_ids.

WIP. 🚧

---

Installation

1. Install PHP8.2+, MongoDB, MongoDB PHP Driver, Composer
2. git clone https://github.com/Hiyori-API/hiyori-cli.git
3. cd hiyori-cli && composer install
4. chmod +x hiyori
5. Run commands

Ingestion

You can use the cli tool as ./hiyori, php hiyori or php src/run.php.

Note

It's recommended to keep a 1-second delay between requests for sources to prevent rate-limiting.

MyAnimeList Ingestion

php hiyori ingest myanimelist --delay 1

Kitsu Ingestion

php hiyori ingest kitsu --delay 1

AniList Ingestion

php hiyori ingest anilist --delay 1

Note

1: General spoiler or Media spoiler tags are not ingested.

2: The total entries shown will max out at 5000. This is just a data issue with the data returned It will not affect the ingestion process as the value will be updated after paginating past 5000 You can find out more about this known-issue on AniList's discord server.

---

Combiner

Note

Run the combiner after the ingestion of multiple sources are complete.

php hiyori combine {base} --strategy {strategy}

• {base} the name of the source to dataset to use
• There are multiple combining strategies available. The default one is relational_mapping.

// example
php hiyori combine myanimelist
// specify a stratgegy
php hiyori combine myanimelist --strategy relational_mapping

Available Combining Strategies

Relational Mapping --strategy relational_mapping

This strategy simply checks for matching source IDs across available source datasets and combines values based off of those.

Relational Fuzzy --strategy relational_fuzzy

This strategy cross-checks all reference_ids across available source datasets in hopes to find a matching ID between sources.

Metadata Fuzzy

Warning

This is not yet implemented.

This strategy compares other metadata across available source datasets. The metadata properties are given weights and are cross checked using various algorithms. Based on this, a score is given. If the score is above a certain threshold, it is merged.

---

Hiyori Schema

Common data with different representations like status (MAL: "Finished Airing", Anilist: "Finished") are transformed into a common value via Hiyori's Enums.

Anime

Property	Data Type	Remarks	Nullable
title	String	Entry's main title	❌
synonyms	Array of String	All other titles (combined from sources)	❌
type	String	Entry type [tv, movie, ova, ona, special, music]	✅
episodes	Integer	Number of episodes (if any mentioned)	✅
status	String	Status of entry [finished, airing, to_be_aired, cancelled]	✅
season	Nullable String	Release Season [winter, summer, spring, fall]	✅
year	Nullable Integer	Release Year	✅
images	Array of String	Default Image URLs (combined from sources)	❌
reference_ids	Object	Parsed IDs from Supported References	❌
references	Array of String	Reference URLs (combined from sources)	❌
tags	Array of String	Genres/Tags (combined from sources)	❌

Manga

TBD.

---

Supported References

The following URL type, if detected in an entry's references, will be parsed.

Reference	Shorthand (as returned)
Amazon Prime	ap
AniDB	adb
AniList	al
AnimeNewsNetwork	ann
Bangumi	b
Crunchyroll	cr
Funimation	fm
Kitsu	k
MyAnimeList	mal
Netflix	nf
Syoboi	sb
Trakt	t
TubiTV	ttv
TVTokyo	tvt
Twitter	tx
Wikipedia English	wpen
Wikipedia Japanese	wpjp
YouTube	yt

---

Roadmap

👉 View Roadmap

FAQ

Click to expand

Why does Hiyori Schema consist of metadata?

In the event an entry from a source does not contain relational IDs or mapping provided by sources, the metadata will be used to find the relation instead.

Will you add more to the metadata upon request?

We're only using metadata to create a relation score if there's no direct reference found between entries. There are no plans to add more metadata unless it can somehow help create a better relation score for unmapped entries.

---

Important

Hiyori is not affiliated with any sources. You are responsible for the usage of this tool. Please be respectful of the terms and conditions set by these sources.