superset db upgrade not working due to removal of FLAGS variable in sqlparse/kewords.py

[LeonardSchuler]

In superset/sql_parse.py sqlparse.keywords.FLAGS is used. However, sqlparse has removed the FLAGS variable from the code sqlparse/kewords.py.
This leads to an AttributeError when running the CLI command
$ superset db upgrade
AttributeError: module 'sqlparse.keywords' has no attribute 'FLAGS'
I would recommend replacing the FLAGS dependency with the actual value
re.IGNORECASE | re.UNICODE).match
or fix the sqlparse version to "sqlparse==0.4.3" (not recommended as they fixed a security problem recently).

I have experienced this problem under python3.9 and python3.11

[Tafkas]

I have the same issue using Python 3.8.16.

[esevato]

Clean install Debian 11 fails @ "superset db upgrade" with
AttributeError: module 'sqlparse.keywords' has no attribute 'FLAGS'

[amilcarnetto]

downgrading to sqlparse 0.4.3 solve the problem for me

[MoronGamer]

downgrading to sqlparse 0.4.3 solve the problem for me

Having same error, how do you downgrade sqlparse?

[HoncyLee]

downgrading to sqlparse 0.4.3 solve the problem for me

Having same error, how do you downgrade sqlparse?

pip install sqlparse=='0.4.3'

[Akshay577]

Yes, downgrading the sqlparse version to 0.4.3 fixed the issue for me.

[anishone]

Downgrading with pip install sqlparse=='0.4.3' works. Thanks!

[SUGAM-ARORA]

downgrading to sqlparse 0.4.3 solve the problem for me

It didn't work for me .

[rohitkumar0613]

Same didn't work for me

[christianxiaobai]

Downgrading with pip install sqlparse=='0.4.3' works. Thanks!

[kits-ragh]

After downgrading sqlparse to 0.4.3 getting following error.

Error: Could not locate a Flask application. You did not provide the "FLASK_APP" environment variable, and a "wsgi.py" or "app.py" module was not found in the current directory.

[pierrelafortune]

+1 to the same FLASK APP error as @kits-ragh

Error: Could not locate a Flask application. You did not provide the "FLASK_APP" environment variable, and a "wsgi.py" or "app.py" module was not found in the current directory.

[prb219]

@LeonardSchuler Thanks for the suggestions. I tried your suggested changes on the following setup but it fails
Superset version: 2.1.0
Python version: 3.8
SQLparse version: 0.4.4

No issue is observed during the db upgrade process. However, when we try to load the dashboards we see this error
`TypeError: first argument must be string or compiled pattern
2023-07-13 21:03:58,523:ERROR:superset.views.base:first argument must be string or compiled pattern
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1517, in full_dispatch_request
rv = self.dispatch_request()
.
.
.
.
File "/usr/local/lib/python3.8/site-packages/sqlparse/engine/filter_stack.py", line 26, in run
stream = lexer.tokenize(sql, encoding)
File "/usr/local/lib/python3.8/site-packages/sqlparse/lexer.py", line 155, in tokenize
return Lexer.get_default_instance().get_tokens(sql, encoding)
File "/usr/local/lib/python3.8/site-packages/sqlparse/lexer.py", line 52, in get_default_instance
cls._default_intance.default_initialization()
File "/usr/local/lib/python3.8/site-packages/sqlparse/lexer.py", line 59, in default_initialization
self.set_SQL_REGEX(keywords.SQL_REGEX)
File "/usr/local/lib/python3.8/site-packages/sqlparse/lexer.py", line 78, in set_SQL_REGEX
self._SQL_REGEX = [
File "/usr/local/lib/python3.8/site-packages/sqlparse/lexer.py", line 79, in
(re.compile(rx, FLAGS).match, tt)
File "/usr/lib64/python3.8/re.py", line 252, in compile
return _compile(pattern, flags)
File "/usr/lib64/python3.8/re.py", line 303, in _compile
raise TypeError("first argument must be string or compiled pattern")
TypeError: first argument must be string or compiled pattern

`

[enoxos]

If you downgraded to sqlparser=0.4.3 and you get the error:

_Error: Could not locate a Flask application. You did not provide the "FLASK_APP" environment variable, and a "wsgi.py" or "app.py" module was not found in the current directory.

then set the FLASK_APP as follows

FLASK_APP=superset

and then rerun the db initialization step.

[mdanishu]

I'm still getting this flags problem... why is this happening and how can we fix it? downgrading to lower versions didn't fix it

[WaluQ-iu]

I'm still getting this flags problem... why is this happening and how can we fix it? downgrading to lower versions didn't fix it

Try export FLASK_APP=superset and then rerun

[helpwise]

/home/ubuntu/.local/bin# sudo -u ubuntu FLASK_APP=superset /home/ubuntu/.local/bin/superset db upgrade

Traceback (most recent call last):
File "/home/ubuntu/.local/bin/superset", line 5, in
from superset.cli.main import superset
File "/home/ubuntu/.local/lib/python3.10/site-packages/superset/init.py", line 21, in
from superset.app import create_app
File "/home/ubuntu/.local/lib/python3.10/site-packages/superset/app.py", line 23, in
from superset.initialization import SupersetAppInitializer
File "/home/ubuntu/.local/lib/python3.10/site-packages/superset/initialization/init.py", line 33, in
from superset.extensions import (
File "/home/ubuntu/.local/lib/python3.10/site-packages/superset/extensions/init.py", line 32, in
from superset.utils.async_query_manager import AsyncQueryManager
File "/home/ubuntu/.local/lib/python3.10/site-packages/superset/utils/async_query_manager.py", line 26, in
from superset.utils.core import get_user_id
File "/home/ubuntu/.local/lib/python3.10/site-packages/superset/utils/core.py", line 106, in
from superset.sql_parse import sanitize_clause
File "/home/ubuntu/.local/lib/python3.10/site-packages/superset/sql_parse.py", line 67, in
re.compile(r"'(''|\\|\|[^'])*'", sqlparse.keywords.FLAGS).match,
AttributeError: module 'sqlparse.keywords' has no attribute 'FLAGS'

FLASK_APP=superset doesnt work for me either..

I am running Ubuntu 20.04 in Proxmox

[WaluQ-iu]

/home/ubuntu/.local/bin# sudo -u ubuntu FLASK_APP=superset /home/ubuntu/.local/bin/superset db upgrade

Traceback (most recent call last): File "/home/ubuntu/.local/bin/superset", line 5, in from superset.cli.main import superset File "/home/ubuntu/.local/lib/python3.10/site-packages/superset/init.py", line 21, in from superset.app import create_app File "/home/ubuntu/.local/lib/python3.10/site-packages/superset/app.py", line 23, in from superset.initialization import SupersetAppInitializer File "/home/ubuntu/.local/lib/python3.10/site-packages/superset/initialization/init.py", line 33, in from superset.extensions import ( File "/home/ubuntu/.local/lib/python3.10/site-packages/superset/extensions/init.py", line 32, in from superset.utils.async_query_manager import AsyncQueryManager File "/home/ubuntu/.local/lib/python3.10/site-packages/superset/utils/async_query_manager.py", line 26, in from superset.utils.core import get_user_id File "/home/ubuntu/.local/lib/python3.10/site-packages/superset/utils/core.py", line 106, in from superset.sql_parse import sanitize_clause File "/home/ubuntu/.local/lib/python3.10/site-packages/superset/sql_parse.py", line 67, in re.compile(r"'(''|\||[^'])*'", sqlparse.keywords.FLAGS).match, AttributeError: module 'sqlparse.keywords' has no attribute 'FLAGS'

AttributeError: module 'sqlparse.keywords' has no attribute 'FLAGS' suggests a compatibility issue or version mismatch with the libraries or modules being used by Superset. Specifically, it mentions an AttributeError related to the FLAGS attribute in the sqlparse.keywords module. Comments above state that you need to downgrade sqlparse to a lower version, check comments above.

[EJ-Humn]

In superset/sql_parse.py sqlparse.keywords.FLAGS is used. However, sqlparse has removed the FLAGS variable from the code sqlparse/kewords.py. This leads to an AttributeError when running the CLI command $ superset db upgrade AttributeError: module 'sqlparse.keywords' has no attribute 'FLAGS' I would recommend replacing the FLAGS dependency with the actual value re.IGNORECASE | re.UNICODE).match or fix the sqlparse version to "sqlparse==0.4.3" (not recommended as they fixed a security problem recently).

I have experienced this problem under python3.9 and python3.11

This worked for me:

re.compile(r"'(''|\\\\|\\|[^'])*'", re.IGNORECASE | re.UNICODE).match,