Provide a set of SQL functions

[Josh Smeaton]

When #14030 lands there will be an easy way to implement SQL functions like LENGTH() and COALESCE(). There are several tickets requesting the implementation of certain functions, and this ticket aims to collate those requests, and extend the number of functions to a useable minimum subset. There was a discussion on the mailing list that calls out some implementation details as well as the set of functions to implement: ​https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!msg/django-developers/HggiPzwkono/SgXt4iNjTIYJ

List of functions:

• Coalesce
• ToLower/Lower
• ToUpper/Upper
• Length
• Case-When / IIF
• Substring
• As/Cast for casting the return value of a complex expression without defining the output_field of each sub-component)

This list isn't necessarily complete or final. If there are other functions that you'd like to see implemented in core, please comment, and I'll update the list above. Particular implementation details can be discussed on the PR or below once a PR is submitted. I'll aim to have these completed very soon after #14030 lands, and I expect these to be available in the same django version that incorporates 14030.

Linked tickets:

• #23363 (Length)
• #21208 (Coalesce in .extra)
• #10929 (Coalesce for Aggregates - could also be done by customising the aggregate directly)

[Josh Smeaton]

I've begun working on this ticket at ​https://github.com/jarshwah/django/tree/ticket_23753

Initial implementation of coalesce and a single very basic test. I'll be adding extra tests and more functions over the coming couple of weeks. When it's getting close to merge-able I'll open a PR for comments. But feel free to make comments before then too.

[Josh Smeaton]

I've implemented the following so far:

• Coalesce
• Concat (accepts 2 params only)
• ConcatMany (same as concat, but can ConcatMany(a,b,c,d)
• Length
• Lower
• Substr
• Upper

Still need to do Case/When, and possibly the As/Cast wrapper, which simply assigns an output_field. Should be about a week away until Case/When is done and docs begin to be written.

[Josh Smeaton]

I've decided against an As wrapper (whose sole role would be to provide an output_field for multiple expressions), because I couldn't really find a need for it in core. We can revisit this decision in the future if users ask for it.

Case/When expressions are currently being worked on by Michal as discussed on the ML. I'll create a separate ticket to track that.

Otherwise, this PR is ready for a full review.

[Josh Smeaton <josh.smeaton@…>]

In 47182965465f47657cbab6858a6a8637cc32b2df:

Fixed #23753 -- Added a suite of SQL Functions

Added functions and tests
Added docs and more tests
Added TextField converter to mysql backend
Aliased Value as V in example docs and tests
Removed unicode_compatible in example
Fixed console emulation in examples

[Tim Graham]

There's a test failure on Oracle/Python 2:

db_functions.tests.FunctionTests.test_coalesce_mixed_values

Traceback (most recent call last):
  File "/mnt/jenkinsdata/workspace/django-oracle/database/oracle11/python/python3.3/django/db/backends/utils.py", line 65, in execute
    return self.cursor.execute(sql, params)
  File "/mnt/jenkinsdata/workspace/django-oracle/database/oracle11/python/python3.3/django/db/backends/oracle/base.py", line 942, in execute
    return self.cursor.execute(query, self._param_generator(params))
cx_Oracle.DatabaseError: ORA-00932: inconsistent datatypes: expected NCHAR got NCLOB

[Josh Smeaton]

The problem here is that NCLOB (TextField) and NCHAR (CharField) are too different in Oracle terms, so the output of the expression with regards to Oracle is not defined.

We should be able to solve this by wrapping CharField parts of the expression in a TO_NCLOB(). If determining whether or not something is a CharField is too difficult, then we can just wrap each expression part inside a TO_NCLOB if self.output_field is a TextField.

I'll also add some additional tests which combine CharFields and TextFields, where the CharFields are other expressions - like Lower('charfield').

[Josh Smeaton]

I implemented the solution above, which passes on my local oracle install.

​https://github.com/django/django/pull/3806

[Josh Smeaton <josh.smeaton@…>]

In 1cdfbde4db1ac6bbddb2cf8dd5192831bc0f9ce3:

Fixed #23753 -- Oracle failure with Coalesce