AttributeError: 'Connection' object has no attribute 'cursor'

[goulashsoup]

If you get this error check the pandas version installed, mine was 2.2.0, with 1.4.3 it worked.

Unfortunately, the package does not restrict pandas to < 2.

@FlipperPA Pls fix that.

[lukestein]

Can also confirm that I was getting this same "'Connection' object has no attribute 'cursor'" error in my normal Python environment with a Pandas version of 2.2.0 and the error seems not to be present using Pandas 1.5.3. I'm hoping there's a better way than this

For reference, this setup seems to give me a working wrds:

conda create -n wrds -c conda-forge python=3.11 anaconda pandas=1 pandas-datareader pandas psycopg2 psycopg2-binary yfinance pip-tools nb_conda_kernels
ponda activate wrds
pip install wrds

[FlipperPA]

@lukestein I'm not an Anaconda user, but I'm pretty sure you should only install psycopg2 OR psycopg2-binary.

@goulashsoup When you get a chance, could you please include a full stack trace so we can see the source of the error? Thank you!

[lukestein]

Here's a trace (hopefully useful); this is using pandas 2.2.0. If I use an install with pandas 1.5.3 then the call to db.raw_sql() runs successfully

{
	"name": "AttributeError",
	"message": "'Connection' object has no attribute 'cursor'",
	"stack": "---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[4], line 1
----> 1 dailystocks = db.raw_sql(\"SELECT cusip, permno, date, bidlo, askhi FROM crsp.dsf LIMIT 100\")

File ~/miniconda3/envs/fdap/lib/python3.11/site-packages/wrds/sql.py:556, in Connection.raw_sql(self, sql, coerce_float, date_cols, index_col, params, chunksize, return_iter)
    499 \"\"\"
    500 Queries the database using a raw SQL string.
    501 
   (...)
    552     2003-09-10  09:35:20.709000  N       AA       None     None  108100.0  28.200          N      00  1.929947e+15         C  None
    553 \"\"\"  # noqa
    555 try:
--> 556     df = pd.read_sql_query(
    557         sql,
    558         self.connection,
    559         coerce_float=coerce_float,
    560         parse_dates=date_cols,
    561         index_col=index_col,
    562         chunksize=chunksize,
    563         params=params,
    564     )
    565     if return_iter or chunksize is None:
    566         return df

File ~/miniconda3/envs/fdap/lib/python3.11/site-packages/pandas/io/sql.py:526, in read_sql_query(sql, con, index_col, coerce_float, params, parse_dates, chunksize, dtype, dtype_backend)
    523 assert dtype_backend is not lib.no_default
    525 with pandasSQL_builder(con) as pandas_sql:
--> 526     return pandas_sql.read_query(
    527         sql,
    528         index_col=index_col,
    529         params=params,
    530         coerce_float=coerce_float,
    531         parse_dates=parse_dates,
    532         chunksize=chunksize,
    533         dtype=dtype,
    534         dtype_backend=dtype_backend,
    535     )

File ~/miniconda3/envs/fdap/lib/python3.11/site-packages/pandas/io/sql.py:2739, in SQLiteDatabase.read_query(self, sql, index_col, coerce_float, parse_dates, params, chunksize, dtype, dtype_backend)
   2728 def read_query(
   2729     self,
   2730     sql,
   (...)
   2737     dtype_backend: DtypeBackend | Literal[\"numpy\"] = \"numpy\",
   2738 ) -> DataFrame | Iterator[DataFrame]:
-> 2739     cursor = self.execute(sql, params)
   2740     columns = [col_desc[0] for col_desc in cursor.description]
   2742     if chunksize is not None:

File ~/miniconda3/envs/fdap/lib/python3.11/site-packages/pandas/io/sql.py:2673, in SQLiteDatabase.execute(self, sql, params)
   2671     raise TypeError(\"Query must be a string unless using sqlalchemy.\")
   2672 args = [] if params is None else [params]
-> 2673 cur = self.con.cursor()
   2674 try:
   2675     cur.execute(sql, *args)

AttributeError: 'Connection' object has no attribute 'cursor'"
}

[FlipperPA]

@lukestein That's interesting, it appears that it is trying to use the SQLite driver instead of PostgreSQL. (with pandasSQL_builder(con) as pandas_sql:)

I'm hoping to release a new version of the package after some more testing tomorrow. The new version (which is committed to the main branch) supports SQLAlchemy 2.0 and pandas 2.2.

[lukestein]

Awesome; thanks. I'll probably wait and try the new release once you've tested rather than trying to pull from main now.

And on your note above about psycopg2/psycopg2-binary I am very much not a conda expert. I had been installing psycopg2 for several years as part of my conda env in order to support wrds, but noted today that pip install wrds still installed psycopg2-binary so figured I'd add it to the conda spec. I'll try to clean that up in any case. Really appreciate your hard work and help on all this.

[FlipperPA]

No sir, thank you! I appreciate the help with bug reports and tracking down edge cases. Very much appreciated!

[FlipperPA]

Hey folks! I've got a version that can be tested now.

On Linux or Mac:

$ python -m venv --copies --prompt wrds-py wrds-py
$ source wrds-py/bin/activate
(wrds-py) $ python -m pip install -U pip wheel ipython
(wrds-py) $ python -m pip install git+https://github.com/wharton/wrds.git@2ca526e

On Windows:

C:\Users\username> python -m venv --copies --prompt wrds-py wrds-py
C:\Users\username> wrds-py\Scripts\activate
(wrds-py) C:\Users\username> python -m pip install -U pip wheel ipython
(wrds-py) C:\Users\username> python -m pip install git+https://github.com/wharton/wrds.git@2ca526e

This installs from GitHub for testing before I publish to PyPI. Let me know if this works, and thanks again for helping sleuth this out, @lukestein and @goulashsoup.

[lukestein]

Seems to be working for me!

My installation was

conda create -n wrds -c conda-forge python=3.11 anaconda pandas-datareader pandas psycopg2-binary yfinance pip-tools nb_conda_kernels
conda activate wrds
pip install git+https://github.com/wharton/wrds.git@2ca526e

Output from the pip install was

DEPRECATION: Loading egg at /Users/lcdstein/miniconda3/envs/wrds/lib/python3.11/site-packages/frozendict-2.3.0-py3.10.egg is deprecated. pip 23.3 will enforce this behaviour change. A possible replacement is to use pip for package installation..
Collecting git+https://github.com/wharton/wrds.git@2ca526e
  Cloning https://github.com/wharton/wrds.git (to revision 2ca526e) to /private/var/folders/cj/g1r1_trj46bc7cjqcjn7l_q80000gn/T/pip-req-build-5y99dq35
  Running command git clone --filter=blob:none --quiet https://github.com/wharton/wrds.git /private/var/folders/cj/g1r1_trj46bc7cjqcjn7l_q80000gn/T/pip-req-build-5y99dq35
  WARNING: Did not find branch or tag '2ca526e', assuming revision or ref.
  Running command git checkout -q 2ca526e
  Resolved https://github.com/wharton/wrds.git to commit 2ca526e
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: numpy<1.27 in ./miniconda3/envs/wrds/lib/python3.11/site-packages (from wrds==3.1.7.dev19+g2ca526e) (1.24.3)
Requirement already satisfied: packaging<23.3 in ./miniconda3/envs/wrds/lib/python3.11/site-packages (from wrds==3.1.7.dev19+g2ca526e) (23.1)
Requirement already satisfied: pandas<2.3 in ./miniconda3/envs/wrds/lib/python3.11/site-packages (from wrds==3.1.7.dev19+g2ca526e) (2.0.3)
Requirement already satisfied: psycopg2-binary<2.10 in ./miniconda3/envs/wrds/lib/python3.11/site-packages (from wrds==3.1.7.dev19+g2ca526e) (2.9.9)
Requirement already satisfied: scipy<1.13 in ./miniconda3/envs/wrds/lib/python3.11/site-packages (from wrds==3.1.7.dev19+g2ca526e) (1.11.1)
Requirement already satisfied: sqlalchemy<2.1 in ./miniconda3/envs/wrds/lib/python3.11/site-packages (from wrds==3.1.7.dev19+g2ca526e) (1.4.39)
Requirement already satisfied: python-dateutil>=2.8.2 in ./miniconda3/envs/wrds/lib/python3.11/site-packages (from pandas<2.3->wrds==3.1.7.dev19+g2ca526e) (2.8.2)
Requirement already satisfied: pytz>=2020.1 in ./miniconda3/envs/wrds/lib/python3.11/site-packages (from pandas<2.3->wrds==3.1.7.dev19+g2ca526e) (2023.3.post1)
Requirement already satisfied: tzdata>=2022.1 in ./miniconda3/envs/wrds/lib/python3.11/site-packages (from pandas<2.3->wrds==3.1.7.dev19+g2ca526e) (2023.3)
Requirement already satisfied: psycopg2==2.9.9 in ./miniconda3/envs/wrds/lib/python3.11/site-packages (from psycopg2-binary<2.10->wrds==3.1.7.dev19+g2ca526e) (2.9.9)
Requirement already satisfied: six>=1.5 in ./miniconda3/envs/wrds/lib/python3.11/site-packages (from python-dateutil>=2.8.2->pandas<2.3->wrds==3.1.7.dev19+g2ca526e) (1.16.0)
Building wheels for collected packages: wrds
  Building wheel for wrds (pyproject.toml) ... done
  Created wheel for wrds: filename=wrds-3.1.7.dev19+g2ca526e-py3-none-any.whl size=11831 sha256=77985535764290199e87aa294d23c41d7bd67512fc5018a7f5e9752cd6e8c784
  Stored in directory: /private/var/folders/cj/g1r1_trj46bc7cjqcjn7l_q80000gn/T/pip-ephem-wheel-cache-a_8d8o2b/wheels/1e/9e/f4/612bc7fa09330fede76d9ea9a48eb53d8dc4df9f7ba2e554ff
Successfully built wrds
Installing collected packages: wrds
Successfully installed wrds-3.1.7.dev19+g2ca526e

This seems to be using pandas 2.0.3 and works successfully to get data from WRDS! 🎉

[FlipperPA]

@lukestein Fantastic, I'll press a release now. Thank you!

[FlipperPA]

@lukestein @goulashsoup Give it a try - released as 3.2.0 with instructions on the README. Thanks again!