Opened 17 years ago
Closed 16 years ago
#4924 closed (fixed)
Loading of compressed fixtures
| Reported by: | Owned by: | Jeremy Dunck | |
|---|---|---|---|
| Component: | Core (Serialization) | Version: | dev |
| Severity: | Keywords: | compression, fixtures, feature | |
| Cc: | semente@… | Triage Stage: | Ready for checkin |
| Has patch: | yes | Needs documentation: | no |
| Needs tests: | no | Patch needs improvement: | no |
| Easy pickings: | no | UI/UX: | no |
Description
Fixtures contain a lot of redundancy (in xml or json format), so they compress well. This patch adds detection and loading of bzip2 or gzip compressed fixtures, since these formats are supported natively in Python. It also contains some test cases.
An example of the utility of this patch: in my project, an initial_data.json fixture was around 4Mb and bzip2'ed down to ~150Kb, a much more reasonable size for storing in a repository. I'm sure there are plenty of projects where this is useful.
The tests pass, except for one which fails in the same way the existing fixture test fails.
Attachments (5)
Change History (19)
comment:1 by , 17 years ago
comment:2 by , 17 years ago
| Has patch: | unset |
|---|
comment:3 by , 17 years ago
| Has patch: | set |
|---|
Ok, got past the spam filter now. There's a trac bug which won't let me attach bzip2'ed files. In the tests/modeltests/fixtures/fixtures folder, we add:
fixture4.json.bz2
fixture5.json
fixture5.json.bz2
Fixture 5 is just a copy of fixture 1, to ensure an error gets raised if there is ambiguity as to which fixture file is meant.
comment:4 by , 17 years ago
| Triage Stage: | Unreviewed → Ready for checkin |
|---|
I've got the tests ready to go, but can't attach them either (keep hitting this trac bug), I'll email them on if a core wants them.
comment:5 by , 17 years ago
I noticed a few small problems, and updated the patch to fix them. The problems were:
- It still checked for all the different compression types, even if a unique filename was specified. So fixture5.json.bz2 and fixture5.json would clash, even if you specifically asked for fixture5.json.bz2.
- The test case omitted checking for data which gets loaded from the initial_data fixture.
- The coding style used alternatingCaps for names, instead of using_underscores.
Have a look at the new one, which should be better.
Thanks,
Lars
comment:6 by , 17 years ago
| Triage Stage: | Ready for checkin → Accepted |
|---|
- Patch needs to be updated following the management.py refactor.
- Patch should handle zip files in addition to bz2 and gz.
- More extensive tests are required, including:
- Need to test explicit compression scheme naming, not just discovery.
- Need to check the edge cases where you name a compression scheme, but not a fixture format.
- Documentation of the new capabilities are required.
comment:8 by , 17 years ago
| Owner: | changed from to |
|---|
I'll see what I can do about russellm's suggestions.
comment:10 by , 17 years ago
| Owner: | changed from to |
|---|
comment:11 by , 17 years ago
| Keywords: | feature added |
|---|---|
| Needs documentation: | set |
| Needs tests: | set |
| Patch needs improvement: | set |
by , 16 years ago
| Attachment: | compressed-fixture.tar.gz added |
|---|
Patch and new files for compressed fixture support.
comment:12 by , 16 years ago
| Needs documentation: | unset |
|---|---|
| Needs tests: | unset |
| Owner: | changed from to |
| Patch needs improvement: | unset |
| Status: | new → assigned |
I've uploaded a new patch addressing the concerns russellm raised in his 09/13/07 comment.
comment:13 by , 16 years ago
| Triage Stage: | Accepted → Ready for checkin |
|---|
comment:14 by , 16 years ago
| Resolution: | → fixed |
|---|---|
| Status: | assigned → closed |
Sorry, can't add patch or tests. I'm being spam filtered =(