Is jupysql supported in jupyter lite?

[gmouawad]

Hello is there a way to use jupysql in a jupyter lite notebook?
if yes, is there any documentation or guide document to establish the connection and execute queries?

[edublancas]

jupysql depends on a bunch of packages and I'm unsure if they'll all work on jupyter lite.

did you try already? if you get a chance to try and encounter issues, let us know!

[edublancas]

closing due to inactivity

[gmouawad]

Hello yes, I have tried and it didn't work, that's why I opened the case. Sorry for the inactivity, I was off the past 3 days

[edublancas]

if you can share the error you encounter, that'd be great so people know what to expect

unfortunately, we don't have any plans to support jupyter lite

[gmouawad]

[edublancas]

ok, so the problem you're seeing is because of duckdb, since duckdb has non-python components, it cannot run on jupyter lite.

this isn't actually a jupysql problem (at least not the error you're showing)

[Pancham97]

Hey @edublancas,

I just ran a simple command to load jupysql extension in JupyterLite and it errored with the following:

Click to expand the error stacktrace

Deploy Dash apps for free on Ploomber Cloud! Learn more: https://ploomber.io/s/signup
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
Cell In[5], line 2
      1 await __import__("piplite").install(**{'requirements': ['jupysql']})
----> 2 get_ipython().run_line_magic('load_ext', 'sql')
      3 get_ipython().run_line_magic('config', 'SqlMagic.displaylimit = 5')

File /lib/python3.11/site-packages/IPython/core/interactiveshell.py:2480, in InteractiveShell.run_line_magic(self, magic_name, line, _stack_depth)
   2478     kwargs['local_ns'] = self.get_local_scope(stack_depth)
   2479 with self.builtin_trap:
-> 2480     result = fn(*args, **kwargs)
   2482 # The code below prevents the output from being displayed
   2483 # when using magics with decorator @output_can_be_silenced
   2484 # when the last Python token in the expression is a ';'.
   2485 if getattr(fn, magic.MAGIC_OUTPUT_CAN_BE_SILENCED, False):

File /lib/python3.11/site-packages/IPython/core/magics/extension.py:33, in ExtensionMagics.load_ext(self, module_str)
     31 if not module_str:
     32     raise UsageError('Missing module name.')
---> 33 res = self.shell.extension_manager.load_extension(module_str)
     35 if res == 'already loaded':
     36     print("The %s extension is already loaded. To reload it, use:" % module_str)

File /lib/python3.11/site-packages/IPython/core/extensions.py:62, in ExtensionManager.load_extension(self, module_str)
     55 """Load an IPython extension by its module name.
     56 
     57 Returns the string "already loaded" if the extension is already loaded,
     58 "no load function" if the module doesn't have a load_ipython_extension
     59 function, or None if it succeeded.
     60 """
     61 try:
---> 62     return self._load_extension(module_str)
     63 except ModuleNotFoundError:
     64     if module_str in BUILTINS_EXTS:

File /lib/python3.11/site-packages/IPython/core/extensions.py:79, in ExtensionManager._load_extension(self, module_str)
     77     mod = import_module(module_str)
     78 mod = sys.modules[module_str]
---> 79 if self._call_load_ipython_extension(mod):
     80     self.loaded.add(module_str)
     81 else:

File /lib/python3.11/site-packages/IPython/core/extensions.py:129, in ExtensionManager._call_load_ipython_extension(self, mod)
    127 def _call_load_ipython_extension(self, mod):
    128     if hasattr(mod, 'load_ipython_extension'):
--> 129         mod.load_ipython_extension(self.shell)
    130         return True

File /lib/python3.11/site-packages/sql/magic.py:776, in load_ipython_extension(ip)
    774 def load_ipython_extension(ip):
    775     """Load the magics, this function is executed when the user runs: %load_ext sql"""
--> 776     sql_magic = SqlMagic(ip)
    777     _set_sql_magic(sql_magic)
    778     ip.register_magics(sql_magic)

File /lib/python3.11/site-packages/ploomber_core/telemetry/telemetry.py:733, in Telemetry.log_call.<locals>._log_call.<locals>.wrapper(*args, **kwargs)
    727     success = dict(
    728         action=f"{action_}-success",
    729         total_runtime=str(datetime.datetime.now() - start),
    730         metadata=metadata_success,
    731     )
    732     func._telemetry_success = success
--> 733     self.log_api(**success)
    735 return result

File /lib/python3.11/site-packages/ploomber_core/telemetry/telemetry.py:520, in Telemetry.log_api(self, action, client_time, total_runtime, metadata)
    501 props = {
    502     "event_id": event_id,
    503     "user_id": uid,
   (...)
    516     "metadata": metadata,
    517 }
    519 if is_install:
--> 520     posthog.capture(
    521         distinct_id=uid, event="install_success_indirect", properties=props
    522     )
    524 posthog.capture(distinct_id=uid, event=action, properties=props)

File /lib/python3.11/site-packages/posthog/__init__.py:58, in capture(distinct_id, event, properties, context, timestamp, uuid, groups, send_feature_flags, disable_geoip)
     26 def capture(
     27     distinct_id,  # type: str
     28     event,  # type: str
   (...)
     36 ):
     37     # type: (...) -> Tuple[bool, dict]
     38     """
     39     Capture allows you to capture anything a user does within your system, which you can later use in PostHog to find patterns in usage, work out which features to improve or where people are giving up.
     40 
   (...)
     56     ```
     57     """
---> 58     return _proxy(
     59         "capture",
     60         distinct_id=distinct_id,
     61         event=event,
     62         properties=properties,
     63         context=context,
     64         timestamp=timestamp,
     65         uuid=uuid,
     66         groups=groups,
     67         send_feature_flags=send_feature_flags,
     68         disable_geoip=disable_geoip,
     69     )

File /lib/python3.11/site-packages/posthog/__init__.py:444, in _proxy(method, *args, **kwargs)
    442 global default_client
    443 if not default_client:
--> 444     default_client = Client(
    445         api_key,
    446         host=host,
    447         debug=debug,
    448         on_error=on_error,
    449         send=send,
    450         sync_mode=sync_mode,
    451         personal_api_key=personal_api_key,
    452         project_api_key=project_api_key,
    453         poll_interval=poll_interval,
    454         disabled=disabled,
    455         disable_geoip=disable_geoip,
    456         feature_flags_request_timeout_seconds=feature_flags_request_timeout_seconds,
    457     )
    459 # always set incase user changes it
    460 default_client.disabled = disabled

File /lib/python3.11/site-packages/posthog/client.py:120, in Client.__init__(self, api_key, host, debug, max_queue_size, send, on_error, flush_at, flush_interval, gzip, max_retries, sync_mode, timeout, thread, poll_interval, personal_api_key, project_api_key, disabled, disable_geoip, historical_migration, feature_flags_request_timeout_seconds)
    118 # if we've disabled sending, just don't start the consumer
    119 if send:
--> 120     consumer.start()

File /lib/python311.zip/threading.py:957, in Thread.start(self)
    955     _limbo[self] = self
    956 try:
--> 957     _start_new_thread(self._bootstrap, ())
    958 except Exception:
    959     with _active_limbo_lock:

RuntimeError: can't start new thread

I suspect it is due to the WASM layer, maybe, and it can't create a new thread yet?

This is what I ran btw

%pip install jupysql --quiet
%load_ext sql
%config SqlMagic.displaylimit = 5

[Pancham97]

However, it does work when using Xeus kernel. I mean, it does not really work, work, but it shows a different error.

Click to expand the new stacktrace

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[5], line 1
----> 1 get_ipython().run_line_magic('load_ext', 'sql')

File /lib/python3.11/site-packages/IPython/core/interactiveshell.py:2480, in InteractiveShell.run_line_magic(self, magic_name, line, _stack_depth)
   2478     kwargs['local_ns'] = self.get_local_scope(stack_depth)
   2479 with self.builtin_trap:
-> 2480     result = fn(*args, **kwargs)
   2482 # The code below prevents the output from being displayed
   2483 # when using magics with decorator @output_can_be_silenced
   2484 # when the last Python token in the expression is a ';'.
   2485 if getattr(fn, magic.MAGIC_OUTPUT_CAN_BE_SILENCED, False):

File /lib/python3.11/site-packages/IPython/core/magics/extension.py:33, in ExtensionMagics.load_ext(self, module_str)
     31 if not module_str:
     32     raise UsageError('Missing module name.')
---> 33 res = self.shell.extension_manager.load_extension(module_str)
     35 if res == 'already loaded':
     36     print("The %s extension is already loaded. To reload it, use:" % module_str)

File /lib/python3.11/site-packages/IPython/core/extensions.py:62, in ExtensionManager.load_extension(self, module_str)
     55 """Load an IPython extension by its module name.
     56 
     57 Returns the string "already loaded" if the extension is already loaded,
     58 "no load function" if the module doesn't have a load_ipython_extension
     59 function, or None if it succeeded.
     60 """
     61 try:
---> 62     return self._load_extension(module_str)
     63 except ModuleNotFoundError:
     64     if module_str in BUILTINS_EXTS:

File /lib/python3.11/site-packages/IPython/core/extensions.py:79, in ExtensionManager._load_extension(self, module_str)
     77     mod = import_module(module_str)
     78 mod = sys.modules[module_str]
---> 79 if self._call_load_ipython_extension(mod):
     80     self.loaded.add(module_str)
     81 else:

File /lib/python3.11/site-packages/IPython/core/extensions.py:129, in ExtensionManager._call_load_ipython_extension(self, mod)
    127 def _call_load_ipython_extension(self, mod):
    128     if hasattr(mod, 'load_ipython_extension'):
--> 129         mod.load_ipython_extension(self.shell)
    130         return True

File /lib/python3.11/site-packages/sql/magic.py:509, in load_ipython_extension(ip)
    502 """Load the extension in IPython."""
    504 # this fails in both Firefox and Chrome for OS X.
    505 # I get the error: TypeError: IPython.CodeCell.config_defaults is undefined
    506 
    507 # js = "IPython.CodeCell.config_defaults.highlight_modes['magic_sql'] = {'reg':[[/](http://127.0.0.1:8000/lab/index.html)^%%sql[/](http://127.0.0.1:8000/lab/index.html)]};" # noqa
    508 # display_javascript(js, raw=True)
--> 509 ip.register_magics(SqlMagic)
    510 ip.register_magics(RenderMagic)
    511 ip.register_magics(SqlPlotMagic)

File /lib/python3.11/site-packages/IPython/core/magic.py:456, in MagicsManager.register(self, *magic_objects)
    452     raise ValueError("Class of magics %r was constructed without "
    453                      "the @register_magics class decorator")
    454 if isinstance(m, type):
    455     # If we're given an uninstantiated class
--> 456     m = m(shell=self.shell)
    458 # Now that we have an instance, we can register it and update the
    459 # table of callables
    460 self.registry[m.__class__.__name__] = m

File /lib/python3.11/site-packages/ploomber_core/telemetry/telemetry.py:733, in Telemetry.log_call.<locals>._log_call.<locals>.wrapper(*args, **kwargs)
    727     success = dict(
    728         action=f"{action_}-success",
    729         total_runtime=str(datetime.datetime.now() - start),
    730         metadata=metadata_success,
    731     )
    732     func._telemetry_success = success
--> 733     self.log_api(**success)
    735 return result

File /lib/python3.11/site-packages/ploomber_core/telemetry/telemetry.py:454, in Telemetry.log_api(self, action, client_time, total_runtime, metadata)
    451 (telemetry_enabled, uid, is_install) = _get_telemetry_info()
    453 # Check latest version
--> 454 check_version(self.package_name, self.version)
    456 if self.print_cloud_message:
    457     check_cloud()

File /lib/python3.11/site-packages/ploomber_core/telemetry/telemetry.py:284, in check_version(package_name, version)
    280     return
    282 # check latest version (this is an expensive call since it hits pypi.org)
    283 # so we only ping the server when it's been 2 days
--> 284 latest = get_latest_version(package_name, version)
    286 # If latest version, do nothing
    287 if version == latest:

File /lib/python3.11/site-packages/ploomber_core/telemetry/telemetry.py:226, in get_latest_version(package_name, version)
    221 def get_latest_version(package_name, version):
    222     """
    223     The function checks for the latest available ploomber version
    224     uid file doesn't exist.
    225     """
--> 226     conn = httplib.HTTPSConnection("pypi.org", timeout=1)
    227     try:
    228         conn.request("GET", f"/pypi/{package_name}/json")

AttributeError: module 'http.client' has no attribute 'HTTPSConnection'

Seems like it is due to a lack of support of ssl in Xeus. I found this issue, but it is not resolved yet. Got any suggestions to get it to work?

[edublancas]

looks like our anonymous telemetry is crashing, try disabling it

export PLOOMBER_STATS_ENABLED=false

[Pancham97]

I tried

%set_env PLOOMBER_STATS_ENABLED=false

but it didn't work unfortunately.

[Pancham97]

I also tried this

import os
os.environ["PLOOMBER_STATS_ENABLED"] = "false"

And it didn't work too 😞

UPDATE: I also tried setting an environment variable right in the conda environment by adding the following to the environment.yaml file and yet it did not work:

name: xeus-lite-wasm
channels:
    - https://repo.mamba.pm/emscripten-forge
    - conda-forge
dependencies:
    - ipympl
    - ipywidgets
    - jupysql
    - xeus-python
variables:
  PLOOMBER_STATS_ENABLED: false

[Pancham97]

@edublancas do you have any suggestions on how I could get it to work? I realize JupySQL and JupyterLite are two different things, but I could really use some help 😅

I was hoping to modify the kernel.json file for JupyterLite but after looking at this response from one of the maintainers, I am not entirely sure what to do. I am not too experienced with Jupyter so I could be missing some obvious things. I would appreciate your help.

[edublancas]

JupyterLite runs on WASM, and many projects aren't compatible with it yet, so even if we fix the original issue that you posted, we'll most likely encounter more issues. Unfortunately, unless someone dives deep into the problems, it's hard even to know how difficult it'll be to make JupySQL compatible with JupyterLite, and sadly, the team doesn't have the bandwidth to take on this, as the vast majority of users don't use JupyterLite.