feat: reference docs (#640)

Adds a new `reference/` path that centralizes the logic for creating
reference documentation

## For Python

We can leverage the existing reference docs pipeline thats used within
the `langchain` repo, and adapting it to include docs from more sources
(see langchain-ai/langchain#33066). More details
about how this works are in `reference/python/README.md`.

## For TypeScript

We'll be keeping the existing [typedoc](https://typedoc.org/) library to
render references, but we're onshoring the work of consolidating those
into one reference site within this repo.

The basic process of how that happens is:
* by reading a statically defined list of packages and their locations
(defined in reference/javascript/build.ts`)
* doing a shallow clone of each unique repo + branch
* doing the appropriate install steps so that types can be inferred
properly
* extracting package entrypoints by reading the `exports` key in
`package.json`
* pointing typedoc to add those as entrypoints with the [packages
strategy](https://typedoc.org/documents/Options.Input.html#packages)
* building the typedoc outputs to reference/dist/javascript/

## New `make` targets

- `make build-references` will run scripts to format python + js
reference documentation concurrently (which will end up in
reference/dist/)
- `make preview-references` will run aforementioned build scripts and
start a dev server (this won't work until a vercel project is
configured)

## Vercel Setup

In order for these changes to be reflected on `reference.langchain.com`,
a new Vercel project needs to be created (which can happen once this is
merged).

This was tested against a "test" vercel setup and works as expected,
with both reference targets accessible from their respective subpaths

Author: hntrl

.editorconfig

@@ -27,7 +27,7 @@ indent_size = 4
 
 # JavaScript/TypeScript files
 [*.{js,jsx,ts,tsx}]
-indent_size = 4
+indent_size = 2
 
 # JSON files
 [*.{json,jsonc}]

Makefile

@@ -1,4 +1,4 @@
-.PHONY: dev build format lint test install clean lint_md lint_md_fix mint-broken-links mint-broken-links-all format-check
+.PHONY: dev build format lint test install clean lint_md lint_md_fix mint-broken-links mint-broken-links-all build-references preview-references format-check
 
 dev:
 	@echo "Starting development mode..."
@@ -74,12 +74,25 @@ mint-broken-links-all: build
 	@command -v mint >/dev/null 2>&1 || { echo "Error: mint is not installed. Run 'npm install -g [email protected]'"; exit 1; }
 	@cd build && mint broken-links 2>&1 | python3 ../scripts/filter_broken_links.py
 
+check-pnpm:
+	@command -v pnpm >/dev/null 2>&1 || { echo >&2 "pnpm is not installed. Please install pnpm to proceed (https://pnpm.io/installation)"; exit 1; }
+
+build-references: check-pnpm
+	@echo "Building references..."
+	cd reference && pnpm i && pnpm build
+
+preview-references: check-pnpm
+	@echo "Previewing references..."
+	cd reference && pnpm i && pnpm run preview
+
 help:
 	@echo "Available commands:"
 	@echo "  make dev             - Start development mode with file watching and mint dev"
 	@echo "  make build           - Build documentation to ./build directory"
 	@echo "  make mint-broken-links - Check for broken links in built documentation (excludes integrations)"
 	@echo "  make mint-broken-links-all - Check for broken links in built documentation (includes all directories)"
+	@echo "  make build-references  - Build reference docs"
+	@echo "  make preview-references - Preview reference docs"
 	@echo "  make format          - Format code"
 	@echo "  make lint            - Lint code"
 	@echo "  make lint_md         - Lint markdown files"

README.md

@@ -13,6 +13,7 @@ Documentation changes follow a PR workflow where all tests must pass before merg
 ```text
 src/                  # Source documentation files (edit these)
 build/                # Generated output files (do not edit)
+reference/            # Reference docs pipeline
 pipeline/             # Build pipeline source code
 tests/                # Test files for the pipeline
 Makefile              # Build automation
@@ -75,6 +76,8 @@ Makefile              # Build automation
 - `make build` - Build documentation to `./build` directory
 - `make mint-broken-links` - Check for broken links in built documentation (excludes integrations)
 - `make mint-broken-links-all` - Check for broken links in built documentation (includes all directories)
+- `make build-references` - Build reference docs
+- `make preview-references` - Preview reference docs
 - `make install` - Install all dependencies
 - `make clean` - Remove build artifacts
 - `make test` - Run the test suite

reference/.gitignore

@@ -0,0 +1 @@
+.vercel

reference/javascript/.eslintignore

@@ -0,0 +1,5 @@
+node_modules
+public
+remotes
+dist
+

reference/javascript/.eslintrc.cjs

@@ -0,0 +1,18 @@
+module.exports = {
+  root: true,
+  env: {
+    node: true,
+    es2022: true,
+  },
+  parser: '@typescript-eslint/parser',
+  parserOptions: {
+    ecmaVersion: 'latest',
+    sourceType: 'module',
+  },
+  plugins: ['@typescript-eslint'],
+  extends: ['eslint:recommended', 'plugin:@typescript-eslint/recommended', 'prettier'],
+  rules: {
+    '@typescript-eslint/no-explicit-any': 'off',
+  },
+  ignorePatterns: ['node_modules/', 'public/', 'remotes/', 'dist/'],
+};

reference/javascript/.gitignore

@@ -0,0 +1,11 @@
+*
+node_modules
+!.gitignore
+!.prettierrc
+!.prettierignore
+!eslint.config.mjs
+!build.ts
+!Makefile
+!package.json
+!pnpm-lock.yaml
+!tsconfig.json

reference/javascript/.prettierignore

@@ -0,0 +1,6 @@
+node_modules
+public
+remotes
+dist
+pnpm-lock.yaml
+

reference/javascript/.prettierrc

@@ -0,0 +1,9 @@
+{
+  "semi": true,
+  "singleQuote": true,
+  "tabWidth": 2,
+  "trailingComma": "es5",
+  "printWidth": 100,
+  "bracketSpacing": true,
+  "arrowParens": "always"
+}