Opened 2 years ago

Closed 14 months ago

Last modified 4 weeks ago

#33817 closed New feature (fixed)

Use python-oracledb instead of cx-oracle

Reported by: Alexander Shishenko Owned by: Jingbei Li
Component: Database layer (models, ORM) Version: dev
Severity: Normal Keywords: oracle
Cc: Florian Apolloner, Christopher Jones, Colin Murtaugh, Marti Raudsepp Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

There is a new library for Oracle databases: https://oracle.github.io/python-oracledb/.
It is intended to replace cx-oracle eventually, and it is fully compatible with it. It supports so-called “thin” mode, which can be used without installing Oracle binary client libraries.
That could be beneficial in some scenarios.

Change History (34)

comment:1 by Mariusz Felisiak, 2 years ago

Cc: Florian Apolloner Christopher Jones added
Component: UncategorizedDatabase layer (models, ORM)
Keywords: oracle added
Triage Stage: UnreviewedAccepted
Type: UncategorizedNew feature

Thanks for the report, see the django-developers thread for a discussion of potential ways to move this forward:

I'd prefer adding support for python-oracledb in Django 4.2 and immediately deprecate using cx_Oracle (will be removed in Django 5.1).

comment:2 by Mariusz Felisiak, 2 years ago

Has patch: set
Needs documentation: set
Needs tests: set
Owner: changed from nobody to Jingbei Li
Patch needs improvement: set
Status: newassigned

comment:3 by Colin Murtaugh, 2 years ago

Cc: Colin Murtaugh added

comment:4 by Jingbei Li, 2 years ago

Needs documentation: unset
Needs tests: unset
Patch needs improvement: unset

I've fixed the patch, tests and documentation. And this ticket is now ready for review.

comment:5 by Suraj Shaw, 2 years ago

Triage Stage: AcceptedReady for checkin

comment:6 by Marti Raudsepp, 2 years ago

Cc: Marti Raudsepp added

comment:7 by Mariusz Felisiak, 22 months ago

Needs documentation: set
Needs tests: set
Patch needs improvement: set
Triage Stage: Ready for checkinAccepted

comment:8 by Jingbei Li, 19 months ago

Needs documentation: unset
Patch needs improvement: unset

comment:9 by Suraj Shaw, 19 months ago

Needs tests: unset

comment:10 by Suraj Shaw, 19 months ago

Triage Stage: AcceptedReady for checkin

comment:11 by Mariusz Felisiak, 19 months ago

Patch needs improvement: set
Triage Stage: Ready for checkinAccepted

comment:12 by Suraj Shaw, 18 months ago

Triage Stage: AcceptedReady for checkin

comment:13 by Mariusz Felisiak, 18 months ago

Triage Stage: Ready for checkinAccepted

Suraj, why did you changed the status? Oracle tests still crash with this patch, it's not ready for checkin.

comment:14 by Suraj Shaw, 17 months ago

@Mariusz Felisiak I tested this patch for oracle tests and i am not seeing any error with 19c db as well as 21c. Shall we change the status from accepted to ready for checkin?

in reply to:  14 ; comment:15 by Mariusz Felisiak, 17 months ago

Replying to Suraj Shaw:

@Mariusz Felisiak I tested this patch for oracle tests and i am not seeing any error with 19c db as well as 21c. Shall we change the status from accepted to ready for checkin?

It crashes on our setup, so it's not ready. We cannot permanently break our CI.

in reply to:  15 comment:16 by Suraj Shaw, 17 months ago

Replying to Mariusz Felisiak:

Replying to Suraj Shaw:

@Mariusz Felisiak I tested this patch for oracle tests and i am not seeing any error with 19c db as well as 21c. Shall we change the status from accepted to ready for checkin?

It crashes on our setup, so it's not ready. We cannot permanently break our CI.

In the setup integration it seems to be using 19c db. Shall we look out to upgrade it to 23c. It may or may not resolve the crash issue but it may be needed at some stage anyway.

comment:17 by Mariusz Felisiak, 17 months ago

In the setup integration it seems to be using 19c db. Shall we look out to upgrade it to 23c. It may or may not resolve the crash issue but it may be needed at some stage anyway.

Oracle 19c will be supported 4 more years, so I see no reason to bump it now. First of all it's really time-consuming (at least few days of my work). Secondly Oracle 23c is not officially supported by python-oracledb or cx_Oracle.

in reply to:  17 ; comment:18 by Suraj Shaw, 17 months ago

Replying to Mariusz Felisiak:

In the setup integration it seems to be using 19c db. Shall we look out to upgrade it to 23c. It may or may not resolve the crash issue but it may be needed at some stage anyway.

Oracle 19c will be supported 4 more years, so I see no reason to bump it now. First of all it's really time-consuming (at least few days of my work). Secondly Oracle 23c is not officially supported by python-oracledb or cx_Oracle.

Oracle 23c is officially supported and i guess it may help us to get past the crash impasse.

in reply to:  18 comment:19 by Mariusz Felisiak, 17 months ago

Oracle 23c is officially supported and i guess it may help us to get past the crash impasse.

Again, Oracle 19c will be supported 4 more years, so I see no reason to bump it now. We cannot drop support for it just because we don't know why it crashes with this patch.

comment:20 by Suraj Shaw, 16 months ago

@Mariusz Felisiak Any leads how shall we proceed further to this PR to land it as soon as possible since its quite long we have been waiting for this feature to be present in django.

in reply to:  20 comment:21 by Florian Apolloner, 16 months ago

Replying to Suraj Shaw:

@Mariusz Felisiak Any leads how shall we proceed further to this PR to land it as soon as possible since its quite long we have been waiting for this feature to be present in django.

As Mariusz already pointed out more than once: Oracle 19c is and still will be supported for a while. So you will need to fix the PR for Oracle 19, once that is done it can get reviewed.

comment:22 by Christopher Jones, 16 months ago

It's not easy to fix something that doesn't reproduce, and where log files are not available !

comment:23 by Florian Apolloner, 16 months ago

Which logs would be needed? The CI job logs are public. If you think it is in the Oracle logs, please tell us which logs would be relevant or which config changes would be required to obtain the necessary logs, then it would be easier for the people having access to help you.

Version 0, edited 16 months ago by Florian Apolloner (next)

comment:24 by Christopher Jones, 16 months ago

We (Oracle) can assist on a live session if we can get someone who can handle the Django infrastructure questions. We span a few timezones. What's convenient? Send me email addresses (mine is in my profile https://github.com/cjbj) to invite.

comment:25 by Florian Apolloner, 16 months ago

@MarkusH: Can you or someone else from ops help out here (another option would be to temporarily give me access to the CI boxes again)?

comment:26 by Mariusz Felisiak, 16 months ago

I can take a look but not in June, maybe in July or August. It's not high on my priority list. I've already spent too much time in June keeping the Oracle backend as functional as the others. As far as I'm aware, it's an issue in python-oracledb or one of the libraries it uses. Of course, we will help you (Oracle) with debugging, even considering the fact that you've never helped us maintain the Oracle backend by providing infrastructure, knowledge, or money. You have to wait patiently like everyone else. It's an open-source.

comment:27 by Suraj Shaw, 16 months ago

After following the logs that has been uploaded , it looks like it is crashing inside the code that is displaying all of the variables in the stack. In the debugging code if it can be disabled it may help us to find out what the real issue is.

comment:28 by Mariusz Felisiak, 14 months ago

Patch needs improvement: unset

comment:29 by Mariusz Felisiak, 14 months ago

Triage Stage: AcceptedReady for checkin

comment:30 by Mariusz Felisiak <felisiak.mariusz@…>, 14 months ago

Resolution: fixed
Status: assignedclosed

In 9946f0b:

Fixed #33817 -- Added support for python-oracledb and deprecated cx_Oracle.

comment:31 by Peter Bittner, 5 months ago

AFAICS, this change is implemented for Django 5+ only, for the moment.

We need to run a Django 4.2 LTS application with Oracle on Python 3.9 on a RHEL 8.9 host. We could upgrade to Django 5.0, but it requires Python >= 3.10, which we don't have available.

Are there any plans to backport this change to 4.2 LTS?

comment:32 by Marti Raudsepp, 5 months ago

@Peter I doubt the Django project will help you here.

The python-oracledb documentation contains instructions how to replace cx_Oracle module with oracledb. I used this for some time with Django 4.2 in development environments without problems (but never in production). https://python-oracledb.readthedocs.io/en/latest/user_guide/appendix_c.html#python-frameworks-sql-generators-and-orms

Alternatively, it's possible to extract the django.db.backends.oracle DB backend package from Django 5.0 and use it with Django 4.2. Of course this may be a more complicated task, as there were likely DB backend API changes between 5.0 and 4.2.

comment:33 by Jingbei Li, 5 months ago

@Peter If you are not afraid of breaking things, you can check some earlier commit of that PR.

For example, the very first commit can just be used as a dirty workaround.

From 0ee2e12252c0eb8b735ad99bb834d4122847f246 Mon Sep 17 00:00:00 2001
From: Jingbei Li <i@jingbei.li>
Date: Tue, 12 Jul 2022 14:22:40 +0800
Subject: [PATCH] added oracledb support

---
 django/db/backends/oracle/base.py          | 7 +++++--
 django/db/backends/oracle/introspection.py | 5 ++++-
 2 files changed, 9 insertions(+), 3 deletions(-)

diff --git a/django/db/backends/oracle/base.py b/django/db/backends/oracle/base.py
index 2ccd3bc0286da..a38bf0421c669 100644
--- a/django/db/backends/oracle/base.py
+++ b/django/db/backends/oracle/base.py
@@ -49,9 +49,12 @@ def _setup_environment(environ):
 
 
 try:
-    import cx_Oracle as Database
+    import oracledb as Database
 except ImportError as e:
-    raise ImproperlyConfigured("Error loading cx_Oracle module: %s" % e)
+    try:
+        import cx_Oracle as Database
+    except ImportError as e:
+        raise ImproperlyConfigured("Error loading oracledb or cx_Oracle module: %s" % e)
 
 # Some of these import cx_Oracle, so import them after checking if it's installed.
 from .client import DatabaseClient  # NOQA
diff --git a/django/db/backends/oracle/introspection.py b/django/db/backends/oracle/introspection.py
index ce743b291d74b..d4b10f0afab65 100644
--- a/django/db/backends/oracle/introspection.py
+++ b/django/db/backends/oracle/introspection.py
@@ -1,6 +1,9 @@
 from collections import namedtuple
 
-import cx_Oracle
+try:
+    import oracledb as cx_Oracle
+except ImportError:
+    import cx_Oracle
 
 from django.db import models
 from django.db.backends.base.introspection import BaseDatabaseIntrospection

comment:34 by Sarah Boyce <42296566+sarahboyce@…>, 4 weeks ago

In cdbd3196:

Refs #33817 -- Corrected errors raised when Oracle driver is not installed.

oracledb_any should reraise ImportError instead of raising
ImproperlyConfigured.

Note: See TracTickets for help on using tickets.
Back to Top