Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support javascript/html output #198

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open

Conversation

lane-s
Copy link

@lane-s lane-s commented Nov 6, 2019

Addresses #53 by introducing a new node.js module called web-renderer. This simple module takes html input from stdin, renders it after waiting some time for javascript to run, takes a screenshot of the output, and then crops the screenshot to eliminate extra whitespace. The resulting PNG image is written to stdout.

jupyter-org-client has also been modified to support a new result type: web results. The idea is that passing raw for html content will render it using the new module.

Many improvements need to be made before this is good, but I think the basic idea works.

A few notes:

  1. I changed the directory structure so that the js directory now has a Makefile that calls the widget Makefile and the web-renderer Makefile.
  2. I made sure that I could build my branch using cask package, install using package-install-file, and then use the new feature. I've added the entire web-renderer module directory to the Cask file, so when Cask is used to build the package, node_modules is copied into the archive. This works for local installation, but it could be a problem if the archive built by Cask is going to go on Melpa. I think I might need to exclude node_modules and instead make sure npm install is run on the web-renderer module during the package installation or the first time the user tries to use the new feature.
  3. The whole process of executing a code block and rendering a resulting image is quite slow. I suppose one way to speed it up would be to keep the node process running on a timeout, so multiple calls wouldn't need to start a new node process and a new headless browser every time. Once the user stops requesting images for a while, the process would stop on its own to avoid wasting system resources.
  4. The image is trimmed a bit too close. It would be nice to trim it less or add a bit of a margin back after the trim.
  5. We should look from Chrome in more default directories for various operating systems and also provide an option for the user to specify where their installation is. This could be a configuration variable in emacs that gets passed to the node module as an argument.
  6. We could potentially also support taking the screenshot through Firefox using selenium.
  7. This should also handle the application/javascript MIME type. I think we would just need to insert the code inside of a <script> tag inside the template we already have.

In the future, web results could also insert an xwidget in order to actually run javascript code in emacs as discussed in #135

Some visualization libraries like vega output html/javascript code which must be
run in order to render the visualization. Emacs will eventually be able to
handle this through xwidgets, but in the meantime it's nice to have at least an
image of the visualization in the org-mode frontend.

To accomplish this, we can introduce a web-renderer backend that uses headless
chrome or firefox in order to render the visualization, takes a screenshot, and
then crops the screenshot in order to get the final visualization as a png.

Currently, the backend is not hooked up to the org-client. It also takes a
naive approach, assuming html input and a Chrome installation on macOS. It also
creates an output file in order to demonstrate the intermediate results.

Try running `cd web-renderer && cat test_chart.html | npm start`

The image manipulation could also use some work. The image is trimmed very close
and could use some margins.
If the :raw argument is supplied, html replies from the kernel are sent to the
web renderer. The resulting image is inserted as an image result.
@lane-s
Copy link
Author

lane-s commented Nov 26, 2019

I checked to see if connecting to a running chrome instance speeds things up at all, but it doesn't. I'm guessing that fetching the scripts isn't helping things along, so maybe there could be a mechanism to cache required scripts locally. It seems like even with simple html and no js, chrome takes a while to take the screenshot, so maybe it's not really worth trying to optimize.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant