#+TITLE:    Incrementalist README
#+AUTHOR:   Jan Moringen
#+EMAIL:    jmoringe@techfak.uni-bielefeld.de
#+LANGUAGE: en

#+OPTIONS:  toc:nil num:nil

* Introduction

  The purpose of the Incrementalist library is incremental parsing of
  Common Lisp code that is contained in a [[https://codeberg.org/scymtym/text.editor-buffer][text.editor-buffer]] buffer
  into a syntax tree. The parsing is incremental in the sense that
  after a small change in the buffer text, the syntax tree can usually
  be updated with a small amount of parsing work.

  *This library is under active development. Its ASDF system
  structure, package structure, exported symbols and protocols may
  change at any time. Consult the NEWS file or the "Changelog" section
  of the manual for lists of changes in specific versions.*

  This document only gives a very brief overview and highlights some
  features. Proper documentation can be found in the
  file:documentation directory.

* Usage Overview

  A central concept of the Incrementalist library is the "analyzer"
  which, given a text.editor-buffer buffer, creates and maintains a
  concrete syntax tree that represents the content of the buffer. We
  therefore start by creating and populating a buffer:

  #+BEGIN_SRC lisp :exports both :results value :wrap example
    (defun example-buffer (&key (content "(((1)))"))
      (text.editor-buffer:make-buffer :initial-contents content))

    (example-buffer)
  #+END_SRC

  #+RESULTS:
  #+begin_example
  #<TEXT.EDITOR-BUFFER.STANDARD-BUFFER:BUFFER 1 line 7 items; time 1 {120CAF9583}>
  #<TEXT.EDITOR-BUFFER.STANDARD-LINE:RIGHT-STICKY-CURSOR 0:7 {120CAF94F3}>
  #+end_example

  We can now attach an analyzer to a buffer:

  #+BEGIN_SRC lisp :exports both :results value :wrap example
    (defun example-analyzer (buffer)
      (make-instance 'incrementalist:analyzer :buffer buffer))

    (example-analyzer (example-buffer))
  #+END_SRC

  #+RESULTS:
  #+begin_example
  #<INCREMENTALIST:ANALYZER N/A,N/A {120DCE6BD3}>
  #+end_example

  To obtain the concrete syntax tree, we must update the analyzer
  after attaching it to the buffer and after changes to the contents
  of the buffer:

  #+BEGIN_SRC lisp :exports both :results output :wrap example
    (multiple-value-bind (buffer cursor) (example-buffer :content "(((11)))")
      (let ((analyzer (example-analyzer buffer)))
        (let ((cache (incrementalist:cache analyzer)))
          (format t "Initial")
          (incrementalist:update analyzer)
          (mapc #'print (incrementalist:find-wads-containing-position
                         cache 0 4))
          ;; Delete first and last buffer item and update analyzer
          (text.editor-buffer:delete-item-before cursor)
          (setf (text.editor-buffer:index cursor) 0)
          (text.editor-buffer:delete-item-after cursor)
          (incrementalist:update analyzer)
          (format t "~&After change")
          (mapc #'print (incrementalist:find-wads-containing-position
                         cache 0 3)))))
  #+END_SRC

  #+RESULTS:
  #+begin_example
  Initial
  (0 . #<INCREMENTALIST:ATOM-WAD rel:0[0],3 -> 0,5 raw: 11>)
  (0 . #<INCREMENTALIST:CONS-WAD rel:0[0],2 -> 0,6>)
  (0 . #<INCREMENTALIST:CONS-WAD rel:0[0],1 -> 0,7>)
  (0 . #<INCREMENTALIST:CONS-WAD abs:0[0],0 -> 0,8>)
  After change
  (0 . #<INCREMENTALIST:ATOM-WAD rel:0[0],2 -> 0,4 raw: 11>)
  (0 . #<INCREMENTALIST:CONS-WAD rel:0[0],1 -> 0,5>)
  (0 . #<INCREMENTALIST:CONS-WAD abs:0[0],0 -> 0,6>)
  #+end_example

  Incrementalist handles syntax errors by adding special nodes to the
  concrete syntax tree:

  #+BEGIN_SRC lisp :exports both :results output :wrap example
    (let* ((analyzer (example-analyzer (example-buffer :content "(#\\Foo")))
           (cache (incrementalist:cache analyzer)))
      (incrementalist:update analyzer)
      (let* ((wads (incrementalist:find-wads-containing-position
                    cache 0 2))
             (wad1 (cdr (first wads)))
             (wad2 (cdr (second wads))))
        (format t "wad    ~A~%errors ~:A~2%" wad1 (incrementalist:errors wad1))
        (format t "wad    ~A~%errors ~:A~%" wad2 (incrementalist:errors wad2))))
  #+END_SRC

  #+RESULTS:
  #+begin_example
  wad    #<ATOM-WAD rel:0[0],1 -> 0,6 raw: #\?>
  errors (#<ERROR-WAD rel:0[0],3 -> 0,6 condition: UNKNOWN-CHARACTER-NAME>)

  wad    #<CONS-WAD abs:0[0],0 -> 0,6>
  errors (#<ERROR-WAD rel:0[0],6 -> 0,6 condition: UNTERMINATED-LIST>)
  #+end_example

# Local Variables:
# eval: (load-library "ob-lisp")
# End: