dumpdata: option to fetch related objects

[stanislas.guerra@…]

Hi,

Just started to ​write a patch that add a --related option to the dumpdata command where you can specify the related objects you want to fetch aside the main one using --pks.

for example:

class Article(models.Model):
    headline = models.CharField(max_length=100, default='Default headline')
    pub_date = models.DateTimeField()
    tag = models.ForeignKey("Tag")


class Tag(models.Model):
    name = models.CharField(max_length=100)



class Blog(models.Model):
    name = models.CharField(max_length=100)
    featured = models.ForeignKey(Article, related_name='fixtures_featured_set')
    articles = models.ManyToManyField(Article, blank=True,
                                      related_name='fixtures_articles_set')

Allows you to dump several blogs, the articles and the tag on the articles.

python manage.py dumpdata myapp.Blog --pks=2,3 --related=articles.tag,articles,featured

This is very basic at the moment and there are some limitations:

• If you use the --natural switch, you have to be careful in the related order because objects are not sorted ;
• Objects can appear several times (in the previous example, if the featured article is also in the articles relation it will be present 2 times in the dump).

[Russell Keith-Magee]

Improving Django's serialisation is a task that has been proposed a number of times, and has been the subject of at least one Summer of Code project, and a number of unsuccessful SoC applications.

I strongly advise that if you're planning this to be integrated into trunk, you spend some time discussing your proposal on django-dev, because what you've proposed here isn't likely to get far. Look through the mailing list archives for more details, but in short, I don't think you'll get a lot of support for adding small features to the existing serializer. What is needed is a genuinely configurable serialisation approach, allowing configuration of serialization depth, format, key naming, and so on. We need to solve the *whole* problem, not just address a very specific subset of the problem that you're currently experiencing.

For administrative purposes, I'm comfortable calling this a duplicate of #4656 (since what you're talking about here is a depth=1 subset of the functionality described in that ticket).

[stanislas.guerra@…]

Replying to russellm:

Improving Django's serialisation is a task that has been proposed a number of times, and has been the subject of at least one Summer of Code project, and a number of unsuccessful SoC applications.

I strongly advise that if you're planning this to be integrated into trunk, you spend some time discussing your proposal on django-dev, because what you've proposed here isn't likely to get far. Look through the mailing list archives for more details, but in short, I don't think you'll get a lot of support for adding small features to the existing serializer. What is needed is a genuinely configurable serialisation approach, allowing configuration of serialization depth, format, key naming, and so on. We need to solve the *whole* problem, not just address a very specific subset of the problem that you're currently experiencing.

For administrative purposes, I'm comfortable calling this a duplicate of #4656 (since what you're talking about here is a depth=1 subset of the functionality described in that ticket).

Hi Russelm,

Well, this is not really a duplicate of #4656 since the proposed patch does not care about depth. You can traverse one relation with two levels of depth and another one with five and so on in the same command.
Is it not a common problem when one need to build some fixtures of test - without dumping/loading the whole database or manually picking each dependency - that can't be fixed without rethinking the whole thing? Or maybe I am missing an existing/obvious solution?

[Russell Keith-Magee]

What you're missing is that you're only providing a solution for one, very specific subset of problems, whereas serialisation is a lot more complex than that. What I'm wary of is changing the serialisation format in a way that only solves a very small subset of problems, because every API we add means another feature we need to maintain backwards compatibility with when we *do* get around to fixing serialisation.

Django's current serializers are sufficient to do a dump and reload. If you want to fix dependency evaluation, a patch providing a flag that tells the serializer to unroll model dependencies, but uses the existing format, would be a lot more likely to get accepted, IMHO. However, as you note, there are all sorts of limitations related to ordering, etc.