Under construction
The goal of the Bug Explainer is to provide a case-based help service for novice programmers encountering bugs in their code when learning a new language. Novices don't know what's going wrong, and they often describe their problems in vague or invalid terms, e.g., "member doesn't work!" Hence their attempts to find answers via Google or FAQ searches often fail to find relevant answers.
To get around the ambiguity problem,
- the front-end tells novices to enter concrete details of their problem, rather than their (often incorrect) interpretation of the problem
- a set of rules analyzes the bug report and finds potential explanations
Bug Explainer Front End
The input interface is where a user specifies a problem. The interface should not require any expert knowledge from the user. A report would be something like this:
| Summary | A short text summary of bug. Not useful for the computer. |
|---|---|
| Input | Sample code that produces the bug |
| Expected | the result(s) the user expected to see |
| Actual | the result(s) the user actually saw |
| Notes | the user's explanation about what they think happened |
| Operating System | the operating system the user is on |
| Language | the language, including implementation version the user has |
This format requires the user to be clear about what they tried and what they saw. This is more critical for helping than what they think happened.
Here are two example bug reports, with just the code parts
| Input | (LET ((L '(A B C))) (DELETE 'A L) L) |
|---|---|
| Expected | (B C) |
| Actual | (A B C) |
| Input | (DELETE '(B) '((A) (B) (C)) |
|---|---|
| Expected | ((A) (C)) |
| Actual | ((A) (B) (C)) |
Bug Explainer Analyzer
The analyzer in a case-based system needs to look at the input case, i.e.,
a bug report, and extract key features that might explain the bug.
The clues to
common bugs can be subtle. What users say is often not helpful.
For example, novices may
report that DELETE doesn't work. Sometimes their example
involves
deleting the first element of a list. Sometimes
the problem is trying to delete an object that is a list.
One way to analyze their code for clues is pattern matching. The matcher used in the Lisp Critic is a good fit, since it's been tested on matching many Lisp code constructs.
A pattern that recognizes the "deleting first element" situation is:
| Input | (?CONTAINS (DELETE ?*)) |
|---|---|
| Expected | ((?* X)) |
| Actual | ((? Y) (?* X)) |
| Explanation | Although DELETE can return a list without the first element, the structure of Lisp lists make it impossible to destructively remove the first element of a list. For more, see http://www.cs.cmu.edu/Groups/AI/html/faqs/lang/lisp/part3/faq-doc-4.html |
Note the use of variables across several fields.
Bug Report #1 is an example of a report that this pattern should match.
Bug Report #2 is an example of a report of a bug due to a different
misunderstanding of DELETE. What pattern would work here?