Exception with psycopg3 binary package on Ubuntu 20.04

[goffioul]

After upgrading my app to django-4.2.4, I attempted to upgrade to psycopg3 (3.1.10) using the binary package on my Ubuntu 20.04 server. I installed psycopg3 with pip using pip install psycopg[binary]. But the app then started to throw exceptions in various views, in particular in form views during rendering of a related field (list views seem fine, as well we form views without a related field):

NotSupportedError at [snip]/units/new/

server-side cursors not supported in pipeline mode

The last entry in the backtrace is as follows:

[snip]/lib/python3.8/site-packages/psycopg/server_cursor.py, line 286, in execute
            raise e.NotSupportedError( …
Local vars
Variable	Value
binary	None
kwargs	{}
params	()
query	('SELECT "config_networkprofile"."id", "config_networkprofile"."name", '
	 '"config_networkprofile"."primaryVoip", '
	 '"config_networkprofile"."primarySipProxy" FROM "config_networkprofile" ORDER '
	 'BY "config_networkprofile"."name" ASC')
self	Error in formatting: ValueError: 32653 is not a valid PipelineStatus

What cauth my attention is the formatting error due to invalid/unexpected PipelineStatus value. As this is precompiled binary, I figured this might be due to some ABI incompatibility. I tried pure-python ("psycopg") and local-install ("psycopg[c]") and they both work fine, no exception is thrown. I also don't get any exception if I disable server-side cursors with DISABLE_SERVER_SIDE_CURSORS.

Things that I think are relevant:

• my database uses defaults for ATOMIC_REQUESTS (False) and AUTOCOMMIT (True)
• I don't use transaction pooling, as mentioned in django docs
• I've reported the issue to psycopg project, but they appear to think the problem is on django side and closed the ticket (​https://github.com/psycopg/psycopg/issues/622)
• I've also tried to use server_side_binding, as suggested in the forum, but that didn't help (​https://forum.djangoproject.com/t/error-problem-with-psycopg3-and-server-side-cursors/22964)

[Mariusz Felisiak]

Hi, I don't think you've explained the issue in enough detail to confirm a bug in Django. Please reopen the ticket if you can debug your issue and provide a sample project that reproduces the issue or details about why and where Django is at fault.

[goffioul]

Thank you for the prompt ticket closing. I believe I’ve exhausted all available support channels and will simply leave it at that, using my own local workaround.

[Natalia Bidart]

Replying to goffioul:

Thank you for the prompt ticket closing. I believe I’ve exhausted all available support channels and will simply leave it at that, using my own local workaround.

I tried to reproduce this issue but I couldn't. I tested with Django 4.2.4 and pyscopg 3.1.10 on an Ubuntu 20.04.6 LTS system using the default Python (3.8.10), ran the whole test suite and no test failed, including test_server_side_cursors_setting. If you could provide a minimal reproducer showing how Django is at fault, we would be happy to reopen the ticket and investigate further.

Cheers, Natalia.

[goffioul]

FYI I found the culprit. Django is not at fault.

[Natalia Bidart]

Thank you for letting us know! What was it?