BaseInlineFormSet unable to save existing objects

[Tobias McNulty]

The Error:

Environment:

Request Method: POST
Request URL: http://localhost:8000/caktus-books/ledger/exchange/215/edit/invoice/
Django Version: 1.0-beta_2-SVN-8874
Python Version: 2.5.2
Installed Applications:
['django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.sites',
 'django.contrib.admin',
 'django.contrib.humanize',
 'django.contrib.markup',
 'crm',
 'ledger']
Installed Middleware:
('django.middleware.common.CommonMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.middleware.doc.XViewMiddleware')


Traceback:
File "/home/tobias/caktus/eclipse-workspace/caktus_books/django/core/handlers/base.py" in get_response
  86.                 response = callback(request, *callback_args, **callback_kwargs)
File "/home/tobias/caktus/eclipse-workspace/caktus_books/django/contrib/auth/decorators.py" in __call__
  67.             return self.view_func(request, *args, **kwargs)
File "/home/tobias/caktus/eclipse-workspace/caktus_books/django/db/transaction.py" in _commit_on_success
  238.                 res = func(*args, **kw)
File "/home/tobias/caktus/eclipse-workspace/caktus_books/ledger/views.py" in create_edit_exchange
  88. 			transactions = transaction_formset.save(commit=False)
File "/home/tobias/caktus/eclipse-workspace/caktus_books/django/forms/models.py" in save
  372.         return self.save_existing_objects(commit) + self.save_new_objects(commit)
File "/home/tobias/caktus/eclipse-workspace/caktus_books/django/forms/models.py" in save_existing_objects
  386.             obj = existing_objects[form.cleaned_data[self._pk_field.name]]

Exception Type: KeyError at /caktus-books/ledger/exchange/215/edit/invoice/
Exception Value: None

From the debugger I found that:

self._pk_field.name = 'id'
self._pk_field.name in form.cleaned_data is True
form.cleaned_data[self._pk_field.name] is None

The Formset:

class BaseTransactionFormSet(BaseInlineFormSet):
	@requires_kwarg('exchange_type')
	def __init__(self, *args, **kwargs):
		self.exchange_type = kwargs.pop('exchange_type')
		super(BaseTransactionFormSet, self).__init__(*args, **kwargs)
	
	def add_fields(self, form, index):
		super(BaseTransactionFormSet, self).add_fields(form, index)
		form.fields.pop('project')
		
		form.fields['memo'].widget = forms.TextInput(attrs={'size':'35'})
		form.fields['quantity'].widget = forms.TextInput(attrs={'size':'5'})
		form.fields['amount'].widget = forms.TextInput(attrs={'size':'8'})
		
		et = self.exchange_type
		if et.credit:
			if et.slug == 'invoice':
				qs = ledger.Account.objects.filter(Q(type=et.credit.type) | Q(type='expense'))
			else:
				qs = ledger.Account.objects.filter(type=et.credit.type)
				
			form.fields['credit'].queryset = qs.order_by('number')
		else:
			form.fields.pop('credit')
		
		if et.debit:
			form.fields['debit'].queryset = ledger.Account.objects.filter(type=et.debit.type).order_by('number')
		else:
			form.fields.pop('debit')
		
TransactionFormSet = inlineformset_factory(ledger.Exchange, ledger.Transaction, formset=BaseTransactionFormSet)

The View:

@login_required
@transaction.commit_on_success
def create_edit_exchange(request, exchange_type_slug, exchange_id=None):
	exchange_type = get_object_or_404(ledger.ExchangeType, slug=exchange_type_slug)
	exchange = None
	
	if exchange_id:
		exchange = get_object_or_404(ledger.Exchange, pk=exchange_id, type=exchange_type)
	
	if request.POST:
		exchange_form = ExchangeForm(
			request.POST,
			instance=exchange, 
			exchange_type=exchange_type,
		)
		transaction_formset = TransactionFormSet(
			request.POST,
			instance=exchange, 
			exchange_type=exchange_type,
		)
		
		if exchange_form.is_valid() and transaction_formset.is_valid():
			exchange = exchange_form.save()
			
			transactions = transaction_formset.save(commit=False)
			for transaction in transactions:
				if exchange_type.credit:
					transaction.credit = exchange_form.cleaned_data['credit']
				if exchange_type.debit:
					transaction.debit = exchange_form.cleaned_data['debit']
				transaction.project = exchange_form.cleaned_data['project']
				transaction.save()
			
			request.user.message_set.create(message='%s successfully saved.' % exchange_type)
			return HttpResponseRedirect(reverse('list_exchanges', kwargs={'exchange_type_slug': exchange.type.slug}))
	else:
		exchange_form = ExchangeForm(
			instance=exchange, 
			exchange_type=exchange_type,
		)
		transaction_formset = TransactionFormSet(
			instance=exchange, 
			exchange_type=exchange_type,
		)

	context = {
		'exchange': exchange,
		'exchange_type': exchange_type,
		'exchange_form': exchange_form,
		'transaction_formset': transaction_formset,
	}
	
	return render_to_response('books/ledger/exchange/create.html', context, context_instance=RequestContext(request))

[Tobias McNulty]

This happens only when saving existing related objects with the InlineFormSet. Creating new objects works fine w/the above code.

[Tobias McNulty]

This happens even if I don't override the formset in inlineformset_factory

[Brian Rosner]

Can you also show the models related to this?

[Tobias McNulty]

The problem was that I wasn't printing out {{ transaction_form }} in my template, but I was printing only the fields I needed. So, that explains the absence of cleaned_data["id"]. When I manually print {{ transaction_form.id }} in my template it works fine.

Two suggestions:

1) make is_valid raise the appropriate error if the pk field is missing from request.POST.

2) make a note in the documentation that {{ form.id }} must be included in your template, if you don't use the {{ form }} or {{ formset }} helpers.

[Fugazi]

I had the same problem. Thanks Tobias for the report and "solution".

[gordyt]

possible related problem (can see in admin)

[gordyt]

I just uploaded a self-contained models.py file at the request of Karen T. from ​http://groups.google.com/group/django-users/browse_thread/thread/e2ed78ae41011b75/ this thread. The comments at the top of the file show the error message. It is possibly related to this ticket and, if so, is very easy to reproduce because you don't need to write any views, etc.

Just drop this into an app and file up the admin interface. Add a Book and (using the inline admin interface), add one or more Authors. This should save with no errors. Then, click on the Book entry in the admin interface. This should display just fine and also show the associated Author(s).

You can make a change or not, just click Save again and you should get an error message.

You will notice that both the Book and the Author models are using a custom primary key - a UUIDField that came straight out of the django-extensions project.

This error seems to occur only with using a custom primary key with an inline object. So if Book and Author each have separate admin pages, no errors occur.

Hope this helps!

--gordy

P.S. - At the top of the thread that I referenced I referred to a customized version of the UUIDField -- one that uses native uuid database types if the backend supports it. To eliminate the possibility that my customized UUIDField was causing the problem, I replicated the issue using an unmodified version of the UUIDField and that is what is included in the models.py file.

[jkocherhans]

Part of the problem here is that you're using auto_created=True on your primary key fields. auto_created means that the *field* was auto_created, not that the *value* will be. Taking that off results in another error though, which occurs because a hidden field is not added to the form unless the primary key is auto_created or is an AutoField. I have a fix for both problems in my branch related to #9284.

[jkocherhans]

(In [10190]) Fixed #9284. Fixed #8813. BaseModelFormSet now calls ModelForm.save().
This is backwards-incompatible if you were doing things to 'initial' in BaseModelFormSet.init, or if you relied on the internal _total_form_count or _initial_form_count attributes of BaseFormSet. Those attributes are now public methods.

[bruce@…]

This bug still appears in revision 11620 from svn. Using gordyt's models.py, adding a book with an author and then editing it generates this:

Environment:

Request Method: POST
Request URL: ​http://127.0.0.1:8000/admin/gamedata/book/6fcbefca-b835-11de-80f2-00188b8e3ea8/
Django Version: 1.2 pre-alpha
Python Version: 2.6.2
Installed Applications:
['django.contrib.auth',

'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.sites',
'django.contrib.admin',
'admin.gamedata',
'admin.django_evolution']

Installed Middleware:
('django.middleware.common.CommonMiddleware',

'django.contrib.sessions.middleware.SessionMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware')

Traceback:
File "/usr/lib64/python2.6/site-packages/django/core/handlers/base.py" in get_response

92. response = callback(request, *callback_args, callback_kwargs)

File "/usr/lib64/python2.6/site-packages/django/contrib/admin/options.py" in wrapper

226. return self.admin_site.admin_view(view)(*args, kwargs)

File "/usr/lib64/python2.6/site-packages/django/utils/decorators.py" in call

23. return self.decorator(self.func)(*args, kwargs)

File "/usr/lib64/python2.6/site-packages/django/views/decorators/cache.py" in _wrapped_view_func

70. response = view_func(request, *args, kwargs)

File "/usr/lib64/python2.6/site-packages/django/contrib/admin/sites.py" in inner

186. return view(request, *args, kwargs)

File "/usr/lib64/python2.6/site-packages/django/db/transaction.py" in _commit_on_success

240. res = func(*args, kw)

File "/usr/lib64/python2.6/site-packages/django/contrib/admin/options.py" in change_view

826. instance=new_object, prefix=prefix)

File "/usr/lib64/python2.6/site-packages/django/forms/models.py" in init

724. queryset=qs)

File "/usr/lib64/python2.6/site-packages/django/forms/models.py" in init

459. super(BaseModelFormSet, self).init(defaults)

File "/usr/lib64/python2.6/site-packages/django/forms/formsets.py" in init

44. self._construct_forms()

File "/usr/lib64/python2.6/site-packages/django/forms/formsets.py" in _construct_forms

88. self.forms.append(self._construct_form(i))

File "/usr/lib64/python2.6/site-packages/django/forms/models.py" in _construct_form

737. form = super(BaseInlineFormSet, self)._construct_form(i, kwargs)

File "/usr/lib64/python2.6/site-packages/django/forms/models.py" in _construct_form

475. pk = self.data[pk_key]

File "/usr/lib64/python2.6/site-packages/django/utils/datastructures.py" in getitem

203. raise MultiValueDictKeyError, "Key %r not found in %r" % (key, self)

Exception Type: MultiValueDictKeyError at /admin/gamedata/book/6fcbefca-b835-11de-80f2-00188b8e3ea8/
Exception Value: Key 'author_set-0-id' not found in <QueryDict: {u'author_set-5-book': [u'6fcbefca-b835-11de-80f2-00188b8e3ea8'], u'author_set-5-name': [u''], u'_save': [u'Save'], u'author_set-TOTAL_FORMS': [u'6'], u'author_set-0-book': [u'6fcbefca-b835-11de-80f2-00188b8e3ea8'], u'title': [u'asdkaskldj'], u'author_set-0-name': [u'asldkjasd'], u'author_set-3-name': [u''], u'author_set-2-book': [u'6fcbefca-b835-11de-80f2-00188b8e3ea8'], u'author_set-INITIAL_FORMS': [u'3'], u'author_set-4-name': [u''], u'author_set-2-name': [u'lksjadlasd'], u'author_set-3-book': [u'6fcbefca-b835-11de-80f2-00188b8e3ea8'], u'author_set-1-book': [u'6fcbefca-b835-11de-80f2-00188b8e3ea8'], u'author_set-4-book': [u'6fcbefca-b835-11de-80f2-00188b8e3ea8'], u'author_set-1-name': [u'lkasjdalsda']}>

[jkocherhans]

Replying to bruce@playfirst.com:

This bug still appears in revision 11620 from svn. Using gordyt's models.py, adding a book with an author and then editing it generates this:

The models are wrong. The UUIDField should not have auto_created=True. auto_created=True means the the field was created automatically, and it isn't in this case. If you can reproduce your bug without auto_created=True, please open a new ticket instead of reopening this one and put something like "references #8813" in the description.

[bruce@…]

I reproduce the error with gordy's models.py file, even if I set auto_created=False for both Book and Author.

I will open another bug also.

The error follows.

MultiValueDictKeyError at /admin/gamedata/book/7432e312-d39b-11de-913b-00188b8e3ea8/

Key 'author_set-0-id' not found in <QueryDict: {u'_save': [u'Save'], u'author_set-TOTAL_FORMS': [u'5'], u'author_set-0-book': [u'7432e312-d39b-11de-913b-00188b8e3ea8'], u'title': [u'book1'], u'author_set-0-name': [u'author1'], u'author_set-3-name': [u''], u'author_set-2-book': [u'7432e312-d39b-11de-913b-00188b8e3ea8'], u'author_set-INITIAL_FORMS': [u'2'], u'author_set-4-name': [u''], u'author_set-2-name': [u'asdasdasd'], u'author_set-3-book': [u'7432e312-d39b-11de-913b-00188b8e3ea8'], u'author_set-1-book': [u'7432e312-d39b-11de-913b-00188b8e3ea8'], u'author_set-4-book': [u'7432e312-d39b-11de-913b-00188b8e3ea8'], u'author_set-1-name': [u'authro2']}>

Request Method: POST
Request URL: ​http://127.0.0.1:8000/admin/gamedata/book/7432e312-d39b-11de-913b-00188b8e3ea8/
Exception Type: MultiValueDictKeyError
Exception Value:

Key 'author_set-0-id' not found in <QueryDict: {u'_save': [u'Save'], u'author_set-TOTAL_FORMS': [u'5'], u'author_set-0-book': [u'7432e312-d39b-11de-913b-00188b8e3ea8'], u'title': [u'book1'], u'author_set-0-name': [u'author1'], u'author_set-3-name': [u''], u'author_set-2-book': [u'7432e312-d39b-11de-913b-00188b8e3ea8'], u'author_set-INITIAL_FORMS': [u'2'], u'author_set-4-name': [u''], u'author_set-2-name': [u'asdasdasd'], u'author_set-3-book': [u'7432e312-d39b-11de-913b-00188b8e3ea8'], u'author_set-1-book': [u'7432e312-d39b-11de-913b-00188b8e3ea8'], u'author_set-4-book': [u'7432e312-d39b-11de-913b-00188b8e3ea8'], u'author_set-1-name': [u'authro2']}>

etc... trac is blocking the rest :(

[bruce@…]

related to #12235

[Jacob]

Milestone 1.1 deleted