Enhancing Development and Debugging Experiences

[AdamSobieski]

It can be complex to inspect solvers' outputs (e.g., plans, workflows, natural language, or stories), find issues or concerns, and rapidly navigate to contextual positions in relevant source code, models, domains, and data. This issue is about how to enhance developers' experiences and provide them with the means of navigating from selections of outputs to relevant highlighted selections of source code, models, domains, and data.

Solvers could, in theory, be toggled to produce additional contents alongside their ordinary outputs, these additional contents including mappings from selections of outputs to relevant source code, models, domains, or data.

Natural-language output, for instance, could be hypertext instead of text. Beyond uses of hyperlinks, arbitrary selections of content could map to debugging data. With respect to implementation, perhaps selections would include non-visible document markup elements and attributes, these attaching to linked data. Similarly, visual and diagrammatic representations of plans and workflows could enable navigation from selections to relevant source code, models, domains, and data.

What do you think about providing these capabilities for developers? Developers would, then, be able to more efficiently version software and datasets, including while making use of metrics and evaluation tools on the outputs (e.g., Grammarly or sentiment analysis).

Might planning domain authoring tools or visual IDE's be able to produce content with toggled debug or trace modes? Perhaps IDE's could output from visual representations to output files with such toggleable modes. Might solvers be able to run in such modes?

See also

Traceability, Provenance, Context, State, Stack trace, Tracing, Logging, Profiling, Instrumentation, Debugger, Breakpoint, Stepping

[aiaustin]

Just a small contribution... as we did a lot of work on user interfaces to AI Planners in the mid-1990s. This includes extended interfaces for browsing plans and plan products, along with the rationale behind planning decisions.

O-Plan and I-X/I-Plan both included an ability to plug in any one of a number of viewers for (partial) plans and (elements of) world state... In O-Plan they were called "PlanWorld Viewers" and examples included graphical plan views, world state simulation snapshots and AutoCAD diagrams of items under construction.

Tate, A. and Drabble, B. (1995) O-Plan's PlanWorld Viewers, Proceedings of the Fourteenth UK Special Interest Group on Planning and Scheduling, Wivenhoe House Conference Centre, Essex University, 22-23 November 1995.
https://www.aiai.ed.ac.uk/project/oplan/documents/1995/95-sigplan14-viewers.pdf

In I-X/I-Plan the interface was called "I-Views" and examples included graphical plan UI and mapping reflecting some of the applications.

We experimented with a number of domain knowledge capture methods and Integrated Development Environments for planning (IDEs). We used engineering methodologies such as CommonKADS, QOC (Questions-Options-Criteria), etc.

Nonlin (and its Decision Graph addon), O-Plan and I-Plan were always intended to capture decision rational behind plan decisions whether made by the system or users. A write up on the approaches was done by Steve Polyak whose PhD focussed on this area.

Polyak, S. and Tate, A. (1998) Rationale in Planning: Causality, Dependencies and Decisions, The Knowledge Engineering Review, Vol 13(3), September, pp. 247-262, 1998.
https://www.aiai.ed.ac.uk/project/oplan/documents/1998/98-ker-rationale.pdf