Opened 7 years ago

Last modified 16 months ago

#28805 assigned New feature

Add database functions for regular expressions, e.g. RegexpReplace

Reported by: Joey Wilhelm Owned by: Nick Pope
Component: Database layer (models, ORM) Version: dev
Severity: Normal Keywords:
Cc: Sardorbek Imomaliev Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: yes
Easy pickings: no UI/UX: no

Description

I've created a database function in my own project to utilize REGEXP_REPLACE, and wanted to contribute it upstream. At a quick glance, it appears that this is only available on PostgreSQL and Oracle. So my main question would be, which route would be preferable for inclusion? Should this be added to the PostgreSQL-specific code and let Oracle languish, or would this require the addition of a new feature flag on database backends?

With the former implementation, I have code ready to go. The latter, I would definitely want some guidance.

This is of course all assuming that this feature is desired.

Here is an example usage:

MyModel.objects.annotate(no_letters=RegexpReplace(F('name'), r'[A-Za-z]+', ''))

Change History (16)

comment:1 by Matthew Schinckel, 7 years ago

Personally, I'd create it in the django.contrib.postgres section: there are already other functions in there that you should be able to look at how they are written.

The other alternative is to put it in django.db.models.functions.text, but I'm not sure how to flag that it only works on specific backends there.

You might want to bring this up on the django-developers list, as that sometimes gets a bit more notice.

comment:2 by Matthew Schinckel, 7 years ago

Oh, it is worth pointing out that this is something that should be easy to package up in a reusable manner, since it (probably) won't require any changes, just the addition of a new class.

That class could then be imported from anywhere.

comment:3 by Joey Wilhelm, 7 years ago

Yeah, I was debating the thought of creating some sort of django-postgres-regex package, for this and related functions. But if I could contribute it to core, why not, ya know?

The implementation relatively easy; I based it off, I believe, Substr.

from django.db.models import Func, Value


class RegexpReplace(Func):
    function = 'REGEXP_REPLACE'

    def __init__(self, expression, pattern, replacement, **extra):
        if not hasattr(pattern, 'resolve_expression'):
            if not isinstance(pattern, str):
                raise TypeError("'pattern' must be a string")
            pattern = Value(pattern)
        if not hasattr(replacement, 'resolve_expression'):
            if not isinstance(replacement, str):
                raise TypeError("'replacement' must be a string")
            replacement = Value(replacement)
        expressions = [expression, pattern, replacement]
        super().__init__(*expressions, **extra)

comment:4 by Tim Graham, 7 years ago

Summary: Provide a new database function for RegexpReplaceAdd a database function for RegexpReplace
Triage Stage: UnreviewedAccepted

For a mergable patch, I think we would want both Oracle and PostgreSQL support.

comment:5 by Sardorbek Imomaliev, 6 years ago

Cc: Sardorbek Imomaliev added

comment:6 by Oskar Persson, 5 years ago

Cc: Oskar Persson added

comment:7 by Nick Pope, 5 years ago

Has patch: set
Needs documentation: set
Needs tests: set
Owner: changed from nobody to Nick Pope
Patch needs improvement: set
Status: newassigned

I have a WIP PR.

comment:8 by Nick Pope, 4 years ago

Needs documentation: unset
Needs tests: unset
Patch needs improvement: unset
Summary: Add a database function for RegexpReplaceAdd database functions for regular expressions, e.g. RegexpReplace

I've updated the PR to add support for RegexpStrIndex, RegexpReplace, and RegexpSubstr.

comment:9 by Mariusz Felisiak, 4 years ago

Patch needs improvement: set

comment:10 by Nick Pope, 3 years ago

Patch needs improvement: unset

comment:11 by Mariusz Felisiak, 3 years ago

Patch needs improvement: set

comment:12 by Nick Pope, 18 months ago

Patch needs improvement: unset

comment:13 by Nick Pope, 16 months ago

Patch needs improvement: set

Moving back to PNI. Having a rethink on a few aspects of this patch.

comment:14 by Mariusz Felisiak, 16 months ago

You can also remove code for MySQL <= 8.0.4.

in reply to:  14 comment:15 by Nick Pope, 16 months ago

Replying to Mariusz Felisiak:

You can also remove code for MySQL <= 8.0.4.

Yup. Already done in what I'm working on.

comment:16 by Oskar Persson, 16 months ago

Cc: Oskar Persson removed
Note: See TracTickets for help on using tickets.
Back to Top