Opened 10 years ago
Closed 5 years ago
#24709 closed New feature (duplicate)
ArrayField doesn't support .update() and F() objects
Reported by: | ris | Owned by: | nobody |
---|---|---|---|
Component: | contrib.postgres | Version: | 1.8 |
Severity: | Normal | Keywords: | postgresql contrib arrayfield update |
Cc: | info@… | Triage Stage: | Accepted |
Has patch: | no | Needs documentation: | no |
Needs tests: | no | Patch needs improvement: | no |
Easy pickings: | no | UI/UX: | no |
Description
On a model along the lines of
class ModelA ( models.Model ): name = TextField () name_array = ArrayField ( models.TextField () )
If you perform
>>> ModelA.objects.update ( name_array = [ F ( "name" ) ] )
You end up with
>>> ModelA.objects.all ()[0].name_array [u'F(name)']
I see what you've done there django, but that's not quite what I meant.
Change History (8)
comment:1 by , 10 years ago
Component: | Database layer (models, ORM) → contrib.postgres |
---|---|
Summary: | New contrib.postgres ArrayField doesn't work quite as expected with .update () and F() objects → ArrayField doesn't support .update() and F() objects |
Triage Stage: | Unreviewed → Accepted |
Type: | Bug → New feature |
comment:2 by , 10 years ago
The silent coercion to string is normal behaviour for Django actually in many cases - we actually would like ModelA.objects.update(name_array=[1, 2])
to work (as it would if it were a CharField
.
Supporting F objects like that is a very nice idea, but I'm not sure how possible it is. If we do implement it, it should work for Hstore and Json as well. In these cases there are more complex options - for example you can update just a single key of the Hstore without affecting the rest, which would be a very useful addition.
A simpler option could be to have a distinct API for it - something like ModelA.objects.update(name_array=FArray(0, F(name))
which would generate an update statement like UPDATE app_modela SET name_array[0]=F(name)
. This API could also be a bit clever, and also support ModelA.objects.update(name_array=FArray([F('name')]))
. We would then have similar FJSON and FHstore objects. I'm pretty sure these could be supported without changes to the core ORM by making them look sufficiently like F objects.
comment:3 by , 9 years ago
I have written a similar separate F API for the the comma-separated fields in Django-MySQL - see http://django-mysql.readthedocs.org/en/latest/model_fields.html#listf-expressions . I made a ListF
class for doing single atomic add/remove operations on either end of the list, which is about all that can be done with MySQL's comma-separated-list parsing capabilities.
It is obviously much less involved than Postgres' arrays, but maybe it will inspire. I think add/remove operations are quite useful to consider on top of the 'set element at position X to Y' and 'set array field to [Y]' operations.
comment:4 by , 9 years ago
This seems like a very similar problem to how F() and Sum() evolved separately when they could have been the same with a little (lot..) more work. Marc, could you require that the inputs to Array/HStore be expressions (with an appropriate _parse_expression() to wrap basic values in Value())? If you provide support for expressions only, then getting basic values in comes fairly naturally too.
I haven't looked into the implementation of these fields though, so I don't know how feasible it would be to support.
comment:5 by , 9 years ago
There is more broken here:
ModelA.objects.update(name_array=F("name_array") + 'some item')
This should result in:
... SET "myapp_modela"."name_arry" = array_append("myapp_modela"."name_arry", 'some item') ...
not in:
... SET "myapp_modela"."name_arry" = ("myapp_modela"."name_arry" + 'some item') ...
comment:6 by , 9 years ago
Cc: | added |
---|
comment:7 by , 8 years ago
I'm not that familiar with the internals, so I don't know if my use case helps flesh out ideas for how to implement this, but here's what I have:
I have a score_weights list, which is an array of floating point numbers.Every User has a "scores" field, containing an array with the same number of items. I'd like to sort users by a weighted sum of these scores, with the weights coming from this score_weights list. In other words, the users are ordered by the dot product of their scores array and the score_weights array.
order_expression = -1 * sum([ F("scores__%s" % i) * score_weights[i] for i in xrange(len(crit.score_weights)) ])
Currently, this raises this error:
Traceback (most recent call last): File "<console>", line 1, in <module> File "/Users/frederikcreemers/.virtualenvs/employebla/lib/python2.7/site-packages/django/db/models/query.py", line 256, in __iter__ self._fetch_all() File "/Users/frederikcreemers/.virtualenvs/employebla/lib/python2.7/site-packages/django/db/models/query.py", line 1087, in _fetch_all self._result_cache = list(self.iterator()) File "/Users/frederikcreemers/.virtualenvs/employebla/lib/python2.7/site-packages/django/db/models/query.py", line 54, in __iter__ results = compiler.execute_sql() File "/Users/frederikcreemers/.virtualenvs/employebla/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 824, in execute_sql sql, params = self.as_sql() File "/Users/frederikcreemers/.virtualenvs/employebla/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 369, in as_sql extra_select, order_by, group_by = self.pre_sql_setup() File "/Users/frederikcreemers/.virtualenvs/employebla/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 47, in pre_sql_setup order_by = self.get_order_by() File "/Users/frederikcreemers/.virtualenvs/employebla/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 308, in get_order_by self.query, allow_joins=True, reuse=None) File "/Users/frederikcreemers/.virtualenvs/employebla/lib/python2.7/site-packages/django/db/models/expressions.py", line 209, in resolve_expression for expr in c.get_source_expressions() File "/Users/frederikcreemers/.virtualenvs/employebla/lib/python2.7/site-packages/django/db/models/expressions.py", line 404, in resolve_expression c.rhs = c.rhs.resolve_expression(query, allow_joins, reuse, summarize, for_save) File "/Users/frederikcreemers/.virtualenvs/employebla/lib/python2.7/site-packages/django/db/models/expressions.py", line 403, in resolve_expression c.lhs = c.lhs.resolve_expression(query, allow_joins, reuse, summarize, for_save) File "/Users/frederikcreemers/.virtualenvs/employebla/lib/python2.7/site-packages/django/db/models/expressions.py", line 403, in resolve_expression c.lhs = c.lhs.resolve_expression(query, allow_joins, reuse, summarize, for_save) File "/Users/frederikcreemers/.virtualenvs/employebla/lib/python2.7/site-packages/django/db/models/expressions.py", line 403, in resolve_expression c.lhs = c.lhs.resolve_expression(query, allow_joins, reuse, summarize, for_save) File "/Users/frederikcreemers/.virtualenvs/employebla/lib/python2.7/site-packages/django/db/models/expressions.py", line 403, in resolve_expression c.lhs = c.lhs.resolve_expression(query, allow_joins, reuse, summarize, for_save) File "/Users/frederikcreemers/.virtualenvs/employebla/lib/python2.7/site-packages/django/db/models/expressions.py", line 404, in resolve_expression c.rhs = c.rhs.resolve_expression(query, allow_joins, reuse, summarize, for_save) File "/Users/frederikcreemers/.virtualenvs/employebla/lib/python2.7/site-packages/django/db/models/expressions.py", line 403, in resolve_expression c.lhs = c.lhs.resolve_expression(query, allow_joins, reuse, summarize, for_save) File "/Users/frederikcreemers/.virtualenvs/employebla/lib/python2.7/site-packages/django/db/models/expressions.py", line 463, in resolve_expression return query.resolve_ref(self.name, allow_joins, reuse, summarize) File "/Users/frederikcreemers/.virtualenvs/employebla/lib/python2.7/site-packages/django/db/models/sql/query.py", line 1462, in resolve_ref self.get_initial_alias(), reuse) File "/Users/frederikcreemers/.virtualenvs/employebla/lib/python2.7/site-packages/django/db/models/sql/query.py", line 1402, in setup_joins names, opts, allow_many, fail_on_missing=True) File "/Users/frederikcreemers/.virtualenvs/employebla/lib/python2.7/site-packages/django/db/models/sql/query.py", line 1370, in names_to_path " not permitted." % (names[pos + 1], name)) FieldError: Cannot resolve keyword u'0' into field. Join on 'scores' not permitted.
comment:8 by , 5 years ago
Resolution: | → duplicate |
---|---|
Status: | new → closed |
Duplicate of #26355
(even though this ticket is older, the other one has a pull request which is why I'm closing this one)
I am not sure that F expressions can be supported like that, but if not, it might be worth documenting the restriction and/or throwing a better error message rather than silently coercing to string.