Function tokenize() is not implemented in Metapath processor

[aj-stein-gsa]

User Story

As a developer of Metaschema-based software, in order to write more efficient and easier to read Metapath in my models, I would like this library to implement the function tokenize().

Goals

• Implement the function for Metapath use
• Provide positive correct use of the function in unit tests
• Provide incorrect correct negative use of the function unit tests

Dependencies

NOTE: This function is a little more complex than others requested through GSA/fedramp-automation#734. There are may be multiple dependencies involved. Further discussion is needed with m-j maintainers.

Acceptance Criteria

• All website and readme documentation affected by the changes in this issue have been updated.
• A Pull Request (PR) is submitted that fully addresses the goals of this User Story. This issue is referenced in the PR.
• The CI-CD build process runs without any reported errors on the PR. This can be confirmed by reviewing that all checks have passed in the PR.

Revisions

No response