adds basic syntax highlighting to code snippets (#40)

note: i think these changes make the outputs look a lot better, but
there's a small cost of some additional dependencies (so chatify takes a
little longer to install). i think it's worth it...

example of new formatting:

![Screenshot 2023-07-31 at 12 28 32
AM](https://github.com/ContextLab/chatify/assets/9030494/c1243483-0bb2-472a-927f-50adce8ba39b)

Author: jeremymanning

README.md

@@ -29,9 +29,7 @@ No further setup is required.  To interact with Chatify about any code in the no
 
 ## Customizing Chatify
 
-Chatify is designed to work by default in the free tiers of [Colaboratory](https://colab.research.google.com/) and [Kaggle](https://www.kaggle.com/code) notebooks, and to operate without requiring any additional costs or setup beyond installing and enabling Chatify itself.
-
-Chatify is designed to work on a variety of systems and setups, including the "free" tiers on Google Colaboratory and Kaggle.  For setups with additional resources, it is possible to switch to better-performing or lower-cost models.  Chatify works in CPU-only environments, but is GPU-friendly (for both CUDA-enabled and Metal-enabled systems).  We support any text-generation model on [Hugging Face](https://huggingface.co/models?pipeline_tag=text-generation&sort=trending), Meta's [Llama 2](https://ai.meta.com/resources/models-and-libraries/llama-downloads/) models, and OpenAI's [ChatGPT](https://chat.openai.com/) models (both ChatGPT-3.5 and ChatGPT-4).  Models that run on Hugging Face or OpenAI's servers require either a [Hugging Face API key](https://huggingface.co/docs/api-inference/quicktour#get-your-api-token) or an [OpenAI API key](https://platform.openai.com/signup), respectively.
+Chatify is designed to work by default in the free tiers of [Colaboratory](https://colab.research.google.com/) and [Kaggle](https://www.kaggle.com/code) notebooks, and to operate without requiring any additional costs or setup beyond installing and enabling Chatify itself.  In addition to Colaboratory and Kaggle notebooks, Chatify also supports a variety of other systems and setups, including running locally or on other cloud-based systems.  For setups with additional resources, it is possible to switch to better-performing or lower-cost models.  Chatify works in CPU-only environments, but it is GPU-friendly (for both CUDA-enabled and Metal-enabled systems).  We support any text-generation model on [Hugging Face](https://huggingface.co/models?pipeline_tag=text-generation&sort=trending), Meta's [Llama 2](https://ai.meta.com/resources/models-and-libraries/llama-downloads/) models, and OpenAI's [ChatGPT](https://chat.openai.com/) models (both ChatGPT-3.5 and ChatGPT-4).  Models that run on Hugging Face or OpenAI's servers require either a [Hugging Face API key](https://huggingface.co/docs/api-inference/quicktour#get-your-api-token) or an [OpenAI API key](https://platform.openai.com/signup), respectively.
 
 Once you have your API key(s), if needed, create a `config.yaml` file in the directory where you launch your notebook.  For the OpenAI configuration, replace `<OPANAI API KEY>` with your actual OpenAI API key (with no quotes) and then create a `config.yaml` file with the following contents:
 
@@ -63,7 +61,7 @@ prompts_config:
 
 ### Llama 2 configuration
 
-If you're running your notebook on a well-resourced machine, you can use this config file to get good performance for free!  The 7B and 13B variants of llama 2 both run on the free tier of Google Colaboratory and Kaggle, but the 13B is substantially slower (hence we use the 7B variant by default). Note that using this configuration requires installing the "HuggingFace" dependencies (`pip install chatify[hf]`).
+If you're running your notebook on a well-resourced machine, you can use this config file to get good performance for free!  The 7B and 13B variants of llama 2 both run on the free tier of Google Colaboratory and Kaggle, but the 13B is substantially slower (therefore we recommend the 7B variant if you're using Colaboratory or Kaggle notebooks). Note that using this configuration requires installing the "HuggingFace" dependencies (`pip install chatify[hf]`).
 
 ```yaml
 cache_config:

chatify/main.py

@@ -1,5 +1,4 @@
 import yaml
-import markdown
 
 import pathlib
 import requests
@@ -12,8 +11,7 @@
 from .chains import CreateLLMChain
 from .widgets import option_widget, button_widget, text_widget, thumbs
 
-from .utils import check_dev_config
-
+from .utils import check_dev_config, get_html
 
 @magics_class
 class Chatify(Magics):
@@ -166,7 +164,7 @@ def gpt(self, inputs, prompt):
                 response = requests.post(combined_url, headers=headers, json=data)
                 output = eval(response.content.decode('utf-8'))
 
-        return markdown.markdown(output.replace("\n", "\n\n"))
+        return get_html(output)
 
 
     def update_values(self, *args, **kwargs):

chatify/utils.py

@@ -3,9 +3,41 @@
 import random
 import urllib
 
+from markdown_it import MarkdownIt
+from pygments.formatters import HtmlFormatter
+from pygments.lexers import get_lexer_by_name
+from pygments import highlight
+
 from langchain.llms.base import LLM
 
 
+def highlight_code(code, name, attrs):
+    """Highlight a block of code"""
+
+    lexer = get_lexer_by_name(name)
+    formatter = HtmlFormatter(cssclass='codehilite', linenos='table')
+
+    return highlight(code, lexer, formatter)
+
+def get_html(markdown, code_style='default'):
+    """Return HTML string rendered from markdown source."""
+
+    md = MarkdownIt(
+        "js-default",
+        {
+            "linkify": True,
+            "html": True,
+            "typographer": True,
+            "highlight": highlight_code,
+        },
+    )
+
+    formatter = HtmlFormatter(style=code_style, linenos='table')
+    css = formatter.get_style_defs()
+
+    return f'<head><style>{css}</style></head>' + md.render(markdown)
+
+
 class FakeListLLM(LLM):
     """Fake LLM wrapper for testing purposes.
 

chatify/widgets.py

@@ -72,7 +72,5 @@ def text_widget():
     text : widgets.HTMLMath
         HTMLMath widget for displaying text.
     """
-    text = widgets.HTMLMath(
-        value='', placeholder='', description='', style=dict(font_size='12px')
-    )
+    text = widgets.HTMLMath(value='', placeholder='', description='')
     return text

setup.py

@@ -11,7 +11,7 @@
 with open('HISTORY.rst') as history_file:
     history = history_file.read()
 
-requirements = ['gptcache', 'langchain', 'openai', 'markdown', 'ipywidgets', 'requests']
+requirements = ['gptcache', 'langchain', 'openai', 'markdown', 'ipywidgets', 'requests', 'markdown-it-py[linkify,plugins]', 'pygments']
 extras = ['transformers', 'torch>=2.0', 'tensorflow>=2.0', 'flax', 'einops', 'accelerate', 'xformers', 'bitsandbytes', 'sentencepiece', 'llama-cpp-python']
 
 test_requirements = [