unified-myst (IN-DEVELOPMENT)
unified-myst is a monorepo containing packages for using MyST (Markedly Structured Text), within the unified ecosystem. It will eventually inform myst-spec.
It currently contains packages for:
- Parsing each syntax of the MyST language, from source text, to micromark-tokens
- Directly compiling HTML from the tokens
- Generating the corresponding MDAST-nodes from the tokens, for use with remark
All packages are scoped under the @unified-myst
namespace: https://www.npmjs.com/org/unified-myst
Example
Take the target syntax (name)=
.
The @unified-myst/target-extension
package provides the mystTargetMmarkExt
, mystTargetHtmlExt
, and mystTargetMdastExt
extensions.
First we can parse the target syntax to micromark-tokens:
import { parse } from 'micromark/lib/parse'
import { postprocess } from 'micromark/lib/postprocess'
import { preprocess } from 'micromark/lib/preprocess'
import { mystTargetMmarkExt } from '@unified-myst/target-extension'
const options = { extensions: [mystTargetMmarkExt] }
const events = postprocess(
parse(options).document().write(preprocess()('(name)=', 'utf8', true))
)
console.log(
events.map((event) => {
return [
event[0],
event[1].type,
`${event[1].start.line}.${event[1].start.column}`,
`${event[1].end.line}.${event[1].end.column}`,
]
})
)
// [
// [ 'enter', 'mystTarget', '1.1', '1.8' ],
// [ 'enter', 'mystTargetMarker', '1.1', '1.2' ],
// [ 'exit', 'mystTargetMarker', '1.1', '1.2' ],
// [ 'enter', 'mystTargetString', '1.2', '1.6' ],
// [ 'enter', 'data', '1.2', '1.6' ],
// [ 'exit', 'data', '1.2', '1.6' ],
// [ 'exit', 'mystTargetString', '1.2', '1.6' ],
// [ 'enter', 'mystTargetMarker', '1.6', '1.8' ],
// [ 'exit', 'mystTargetMarker', '1.6', '1.8' ],
// [ 'exit', 'mystTarget', '1.1', '1.8' ]
// ]
Tokens are pointers to slices of the original source text, which can be nested:
These tokens can be compiled directly to HTML:
import { micromark } from 'micromark'
import { mystTargetMmarkExt, mystTargetHtmlExt } from '@unified-myst/target-extension'
const options = {
extensions: [mystTargetMmarkExt],
htmlExtensions: [mystTargetHtmlExt],
}
console.log(micromark('(name)=', options))
// 🔗
Or be compiled to MDAST-nodes:
import { fromMarkdown } from 'mdast-util-from-markdown'
import { mystTargetMmarkExt, mystTargetHtmlExt } from '@unified-myst/target-extension'
const options = {
extensions: [mystTargetMmarkExt],
mdastExtensions: [mystTargetMdastExt],
}
const mdast = fromMarkdown('(name)=', 'utf8', options)
console.log(JSON.stringify(mdast, null, ' '))
// {
// "type": "root",
// "children": [
// {
// "type": "target",
// "position": {
// "start": {
// "line": 1,
// "column": 1,
// "offset": 0
// },
// "end": {
// "line": 1,
// "column": 8,
// "offset": 7
// }
// },
// "label": "name"
// }
// ],
// "position": {
// "start": {
// "line": 1,
// "column": 1,
// "offset": 0
// },
// "end": {
// "line": 1,
// "column": 8,
// "offset": 7
// }
// }
// }
Development
The repository is a monorepo, with shared resources and dependencies. To manage the packages and run script commands against one or more packages, it is advised to use lerna, or using the npm workspace commands.
The repository is currently typed using JSDoc, as opposed to TypeScript. This decision was made, primarily because core packages in the unified ecosystem are also typed in this way, and also just because I wanted to try it out. See this post for a brief comparison. The repository may be converted to TypeScript in the future, but for now it is quite helpful for development, to not require a compilation step.
The repository is formatted with prettier, linted with eslint, and unit tested using Jest. There are a number of JavaScript unit testing frameworks (see this comparison, but jest was chosen because of it is easy to setup/use, flexible, and well used in large projects. See this guide for steps required for ECMAScript Modules (ESM) support. Snapshot Testing is also used extensively; to update snapshots, run lerna run test -- -- -u
or npm run -ws test -- -u
.
rollup.js is used for package bundling. Note that most packages in the unified ecosystem are ESM only
To release new versions of the packages, run lerna publish
.