Repository stabilization, target-neutral ArcIR extraction, immutable ArcIR state persistence, additive semantic enrichment, complete F1 accounting, and the selected-literal curation primitive.
- Added the packable, target-neutral
BioFSharp.ArcIRnetstandard2.0package with validated absolute IRI identities, normalized identity-keyed graph collections, shared term definitions, explicit add/upsert/merge operations, validation, and persistence contracts. - Added canonical
.arcir.json1.0 persistence with a packaged JSON Schema, deterministic UTF-8/LF output, strict versioned decoding, invariant tagged values, authoritative identity keys, atomic create-new state publication, and SHA-256 artifact revisions. - Added typed RFC 6901 fragment locations for every addressable graph entity and atomic value occurrence, including relation-property annotations, plus digest-verified resolution that keeps scalar values ID-less and provenance outside the graph state.
- Added strict inverse parsing for supported ArcIR JSON selectors and the immutable
LiteralMapping.applytransformation for mapping one exact string occurrence to an already registered term while preserving the source literal. - Added format-neutral additive semantic mapping and a
PolyglotSSSOMadapter that preserves complete candidate claims, resolves only declared CURIEs, and leaves mapping selection and provenance to the integrating application. - Added deterministic occurrence-level F1 accounting and resolvable source/output designations for all eight supported INSDC entities plus supplementary paper and count metadata.
- Removed the unused
BioFSharpumbrella-package dependency and replaced the remaining preview dependency with stablePolyglotSSSOM0.1.0. - Converted
BioFSharp.INSDC.ArcIRinto an INSDC-specific F1 adapter. All eight entity converters and supplementary paper/count ingestion now mint deterministic assertion/relation identities and consume the neutral core without retaining the proof-of-concept model shapes. - Added a core-only test project and adapter regression coverage for conflict reporting, reference validation, deterministic identities, complete term resolution, and deterministic eight-entity fixture conversion.
- Limited the nested FAKE solution build to one MSBuild node, preventing runaway worker fan-out and silent process-limit failures in captured Windows builds.
- Made package creation reuse the already verified Release build outputs instead of rebuilding projects and risking locked output assemblies.
- Added dependency-vulnerability and generated-artifact drift gates to the full FAKE test path; generators now produce deterministic committed output.
- Added a versioned, forward-migrated SQLite schema with explicit foreign-key modes, transactional public writes, and exact fixture round-trip coverage for all five stored entities.
- Hardened crawler cancellation, bounded retries, strict partial-failure handling, upstream parsing, atomic writes, validation, and resume behavior while preserving injectable offline fetch seams.
- Split the test suite into focused IO, ArcIR, SQLite, crawler, and crawler-hardening modules; refreshed package-boundary and ArcIR terminology documentation; removed the stale local fsdocs publication path.
- Removed the unused structural-decompilation overlay from the proof-of-concept ArcIR package; its explicit converters remain the current mapping surface.
Crawler fixes surfaced by large-scale crawls (incl. ENA umbrella projects such as PRJNA9506, which fans out to thousands of child projects).
- BioFSharp.INSDC.Crawler — a project/study with no sequencing runs now persists its own record: discovery is run-driven, so a childless root previously discovered nothing (not even itself).
Discovery.withRootseeds the root into the bucket its accession prefix implies (PRJ…→ BioProject,SRP/ERP/DRP…→ Study). - BioFSharp.INSDC.Crawler — new
Startedcrawl event, emitted before discovery, so the first log line identifies the root accession rather than only learning it atdone —. - BioFSharp.INSDC.SQLite / Crawler — bulk insert:
Sql.withTransactionis now reentrant (a nested call joins the active transaction rather than throwing, since SQLite has no nested transactions), letting the crawler persist an entire crawl in a single transaction instead of one commit per record — the difference between a trickle and a bulk load on crawls of hundreds of thousands of runs.
Expands the suite from two packages to five: BioFSharp.FileFormats.INSDC and
BioFSharp.IO.INSDC gain new capabilities, while BioFSharp.INSDC.SQLite,
BioFSharp.INSDC.ArcIR, and BioFSharp.INSDC.Crawler ship for the first time.
- BioFSharp.FileFormats.INSDC — generated per-type XPointer/XPath fragment selectors (
FragmentSelectors.cs) so individual elements of a record can be addressed by fragment identifier. - BioFSharp.IO.INSDC — structural ontology that decompiles records into ontology term/value pairs whose term names mirror the XML structure; fragment-selector tracking via per-instance
xpathOf(bare XPath) andxpointerOf(#xpointer) lookups plus anxpathEntriesDTO. - BioFSharp.INSDC.SQLite (new) — SQLite-backed store that deconstructs BioProject, Study, BioSample, Experiment, and Run records into a normalized schema and reconstructs them on read, with per-entity modules and an
accession_relationstable capturing the cross-record connectivity graph. - BioFSharp.INSDC.ArcIR (new) — maps INSDC records into ArcIR, an ARC-oriented intermediate representation (a property graph of typed, annotations-first objects and relations) with sample references resolved to their BioSample node; renders the graph to GraphML, interactive HTML, and text; ingests supplementary papers and count data.
- BioFSharp.INSDC.Crawler (new) — crawls a project accession from ENA (Portal
filereportdiscovery to Browser API fetch) and persists every connected run, experiment, sample, and study via the SQLite store plus its connectivity table; exposescrawl/crawlToSqlite(with*Async/*WithAsyncvariants); targets net8.0 because FsHttp requires .NET 6+, published to NuGet like the rest.
initial release of the generated C# type model and F# wrapper for the INSDC file formats. The type model covers all schemas from https://ftp.ebi.ac.uk/pub/databases/ena/doc/xsd/sra_1_5, but IO only supports the most common ones for now:
- BioProject
- Study
- BioSample
- Experiment
- Run
- Submission
- Analysis
- Initial repository scaffolding for BioFSharp.INSDC.
- Two-package layout established:
BioFSharp.FileFormats.INSDC(C# generated type model) andBioFSharp.IO.INSDC(F# wrapper).