Use python-oracledb instead of cx-oracle

[Alexander Shishenko]

There is a new library for Oracle databases: ​https://oracle.github.io/python-oracledb/.
It is intended to replace cx-oracle eventually, and it is fully compatible with it. It supports so-called “thin” mode, which can be used without installing Oracle binary client libraries.
That could be beneficial in some scenarios.

[Mariusz Felisiak]

Thanks for the report, see the ​django-developers thread for a discussion of potential ways to move this forward:

I'd prefer adding support for python-oracledb in Django 4.2 and immediately deprecate using cx_Oracle (will be removed in Django 5.1).

[Mariusz Felisiak]

​PR

[Jingbei Li]

I've fixed the patch, tests and documentation. And this ticket is now ready for review.

[Mariusz Felisiak]

Suraj, why did you changed the status? Oracle tests still crash with this patch, it's not ​ready for checkin.

[Suraj Shaw]

@Mariusz Felisiak I tested this patch for oracle tests and i am not seeing any error with 19c db as well as 21c. Shall we change the status from accepted to ready for checkin?

[Mariusz Felisiak]

Replying to Suraj Shaw:

@Mariusz Felisiak I tested this patch for oracle tests and i am not seeing any error with 19c db as well as 21c. Shall we change the status from accepted to ready for checkin?

It crashes on our setup, so it's not ready. We cannot permanently break our CI.

[Suraj Shaw]

Replying to Mariusz Felisiak:

Replying to Suraj Shaw:

@Mariusz Felisiak I tested this patch for oracle tests and i am not seeing any error with 19c db as well as 21c. Shall we change the status from accepted to ready for checkin?

It crashes on our setup, so it's not ready. We cannot permanently break our CI.

In the setup integration it seems to be using 19c db. Shall we look out to upgrade it to 23c. It may or may not resolve the crash issue but it may be needed at some stage anyway.

[Mariusz Felisiak]

In the setup integration it seems to be using 19c db. Shall we look out to upgrade it to 23c. It may or may not resolve the crash issue but it may be needed at some stage anyway.

Oracle 19c will be supported 4 more years, so I see no reason to bump it now. First of all it's really time-consuming (at least few days of my work). Secondly Oracle 23c is not officially supported by python-oracledb or cx_Oracle.

[Suraj Shaw]

Replying to Mariusz Felisiak:

In the setup integration it seems to be using 19c db. Shall we look out to upgrade it to 23c. It may or may not resolve the crash issue but it may be needed at some stage anyway.

Oracle 19c will be supported 4 more years, so I see no reason to bump it now. First of all it's really time-consuming (at least few days of my work). Secondly Oracle 23c is not officially supported by python-oracledb or cx_Oracle.

Oracle 23c is officially supported and i guess it may help us to get past the crash impasse.

[Mariusz Felisiak]

Oracle 23c is officially supported and i guess it may help us to get past the crash impasse.

Again, Oracle 19c will be supported 4 more years, so I see no reason to bump it now. We cannot drop support for it just because we don't know why it crashes with this patch.

[Suraj Shaw]

@Mariusz Felisiak Any leads how shall we proceed further to this PR to land it as soon as possible since its quite long we have been waiting for this feature to be present in django.

[Florian Apolloner]

Replying to Suraj Shaw:

@Mariusz Felisiak Any leads how shall we proceed further to this PR to land it as soon as possible since its quite long we have been waiting for this feature to be present in django.

As Mariusz already pointed out more than once: Oracle 19c is and still will be supported for a while. So you will need to fix the PR for Oracle 19, once that is done it can get reviewed.

[Christopher Jones]

It's not easy to fix something that doesn't reproduce, and where log files are not available !

[Florian Apolloner]

Which logs would be needed? The CI job logs are public. If you think it is in the Oracle logs, please tell us which logs would be relevant or which config changes would be required to obtain the necessary logs, then it would be easier for the people having access to help you.

Maybe we can schedule a live session with someone who has access to the server and someone with knowledge of the oracle changes, so it could be debugged live?

[Christopher Jones]

We (Oracle) can assist on a live session if we can get someone who can handle the Django infrastructure questions. We span a few timezones. What's convenient? Send me email addresses (mine is in my profile ​https://github.com/cjbj) to invite.

[Florian Apolloner]

@MarkusH: Can you or someone else from ops help out here (another option would be to temporarily give me access to the CI boxes again)?

[Mariusz Felisiak]

I can take a look but not in June, maybe in July or August. It's not high on my priority list. I've already spent too much time in June keeping the Oracle backend as functional as the others. As far as I'm aware, it's an issue in python-oracledb or one of the libraries it uses. Of course, we will help you (Oracle) with debugging, even considering the fact that you've never helped us maintain the Oracle backend by providing infrastructure, knowledge, or money. You have to wait patiently like everyone else. It's an open-source.

[Suraj Shaw]

After following the logs that has been uploaded , it looks like it is crashing inside the code that is displaying all of the variables in the stack. In the debugging code if it can be disabled it may help us to find out what the real issue is.

[Mariusz Felisiak <felisiak.mariusz@…>]

In 9946f0b:

Fixed #33817 -- Added support for python-oracledb and deprecated cx_Oracle.

[Peter Bittner]

AFAICS, this change is implemented ​for Django 5+ only, for the moment.

We need to run a Django 4.2 LTS application with Oracle on Python 3.9 on a RHEL 8.9 host. We could upgrade to Django 5.0, but it requires Python >= 3.10, which we don't have available.

Are there any plans to backport this change to 4.2 LTS?

[Marti Raudsepp]

@Peter I doubt the Django project will help you here.

The python-oracledb documentation contains instructions how to replace cx_Oracle module with oracledb. I used this for some time with Django 4.2 in development environments without problems (but never in production). ​https://python-oracledb.readthedocs.io/en/latest/user_guide/appendix_c.html#python-frameworks-sql-generators-and-orms

Alternatively, it's possible to extract the django.db.backends.oracle DB backend package from Django 5.0 and use it with Django 4.2. Of course this may be a more complicated task, as there were likely DB backend API changes between 5.0 and 4.2.

[Jingbei Li]

@Peter If you are not afraid of breaking things, you can check some earlier commit of that PR.

For example, the very first commit can just be used as a dirty workaround.

From 0ee2e12252c0eb8b735ad99bb834d4122847f246 Mon Sep 17 00:00:00 2001
From: Jingbei Li <i@jingbei.li>
Date: Tue, 12 Jul 2022 14:22:40 +0800
Subject: [PATCH] added oracledb support

---
 django/db/backends/oracle/base.py          | 7 +++++--
 django/db/backends/oracle/introspection.py | 5 ++++-
 2 files changed, 9 insertions(+), 3 deletions(-)

diff --git a/django/db/backends/oracle/base.py b/django/db/backends/oracle/base.py
index 2ccd3bc0286da..a38bf0421c669 100644
--- a/django/db/backends/oracle/base.py
+++ b/django/db/backends/oracle/base.py
@@ -49,9 +49,12 @@ def _setup_environment(environ):
 
 
 try:
-    import cx_Oracle as Database
+    import oracledb as Database
 except ImportError as e:
-    raise ImproperlyConfigured("Error loading cx_Oracle module: %s" % e)
+    try:
+        import cx_Oracle as Database
+    except ImportError as e:
+        raise ImproperlyConfigured("Error loading oracledb or cx_Oracle module: %s" % e)
 
 # Some of these import cx_Oracle, so import them after checking if it's installed.
 from .client import DatabaseClient  # NOQA
diff --git a/django/db/backends/oracle/introspection.py b/django/db/backends/oracle/introspection.py
index ce743b291d74b..d4b10f0afab65 100644
--- a/django/db/backends/oracle/introspection.py
+++ b/django/db/backends/oracle/introspection.py
@@ -1,6 +1,9 @@
 from collections import namedtuple
 
-import cx_Oracle
+try:
+    import oracledb as cx_Oracle
+except ImportError:
+    import cx_Oracle
 
 from django.db import models
 from django.db.backends.base.introspection import BaseDatabaseIntrospection

[Sarah Boyce <42296566+sarahboyce@…>]

In cdbd3196:

Refs #33817 -- Corrected errors raised when Oracle driver is not installed.

oracledb_any should reraise ImportError instead of raising
ImproperlyConfigured.