Opened 16 years ago

Closed 12 years ago

Last modified 11 years ago

#9112 closed Bug (fixed)

New releases do not reliably overwrite old ones

Reported by: holdenweb Owned by: packagers
Component: Documentation Version: 1.0
Severity: Release blocker Keywords: release engineering
Cc: krzysiumed@… Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

There have been several issues reported where .pyc files from older releases apparently outlive the installation of a newer release.

The reason for this is that currently releases appear to be cut without altering the modification date of distributed files. Thus a .py file has not been changed since Jan 12, 2007 will appear in the release with that modification date.

When the release is installed with distutils the new .py files will replace their older versions, but if the .pyc from the prior release was compiled *after* the new .py file was last modified in SVN the new file will appear older than its corresponding .pyc, and the compiled file from the prior release will be retained, to the consternation of the installing user.

The simplest fix for this would appear to be to change the release engineering process so that after extraction from SVN all source files are touched to bring their modification date close to that of the release.

Strictly speaking this might also be considered a distutils bug, but the issue has not been reported elsewhere.

Attachments (1)

ticket_9112.diff (1.4 KB ) - added by Christopher Medrela 12 years ago.
Recommending to use pip - draft.

Download all attachments as: .zip

Change History (20)

comment:1 by Karen Tracey, 16 years ago

#9090 is one symptom of this problem. As described there, the problem is not that the .pyc files are not updated but that the newer .py files are not copied to the installation directory. So even clearing out all the .pyc files in the install tree after this has happened won't produce a consistent level, since you've got a mixture of old and new .py files installed. Also, more discussion is here: http://groups.google.com/group/django-developers/browse_thread/thread/85d9500f319369a1. The problem seems to be resulting from old release tarfiles using tarfile creation date for contained files while 1.0 has last-modified-in-svn dates. That makes some files in the 1.0 release appear to be older than what was previously installed. (I do not think there is a distutils bug here.)

comment:2 by (none), 15 years ago

milestone: post-1.0

Milestone post-1.0 deleted

comment:3 by Jacob, 15 years ago

Triage Stage: UnreviewedDesign decision needed

So what's the fix here? Touch all the files as part of rolling the distribution (i.e. from setup.py sdist)?

comment:4 by Luke Plant, 13 years ago

Severity: Normal
Type: Bug

comment:5 by Aymeric Augustin, 12 years ago

Easy pickings: unset
Triage Stage: Design decision neededAccepted
UI/UX: unset

Given Karen's last message in the thread linked above, the safest solution is to ensure all files in the tarball have a modification time set to the release date.

This should be documented in our release process document, once we write it (hopefully, while we release 1.4).

An interesting observation is that svn export gives you the last-modified-in-SVN date, while svn checkout gives you "now".

aaugustin@aaugustin Desktop % svn checkout https://code.djangoproject.com/svn/django/trunk/django/template                           ~/Desktop
(... snip ...)
Checked out revision 17200.
aaugustin@aaugustin Desktop % ls -al template                                                                                        ~/Desktop
total 360
drwxr-xr-x  14 aaugustin  staff    476 12 déc 19:47 ./
drwx------+ 14 aaugustin  staff    476 12 déc 19:47 ../
drwxr-xr-x   9 aaugustin  staff    306 12 déc 19:47 .svn/
-rw-r--r--   1 aaugustin  staff   3247 12 déc 19:47 __init__.py
-rw-r--r--   1 aaugustin  staff  49210 12 déc 19:47 base.py
-rw-r--r--   1 aaugustin  staff   6146 12 déc 19:47 context.py
-rw-r--r--   1 aaugustin  staff   3586 12 déc 19:47 debug.py
-rw-r--r--   1 aaugustin  staff  28093 12 déc 19:47 defaultfilters.py
-rw-r--r--   1 aaugustin  staff  46886 12 déc 19:47 defaulttags.py
-rw-r--r--   1 aaugustin  staff   7886 12 déc 19:47 loader.py
-rw-r--r--   1 aaugustin  staff  10519 12 déc 19:47 loader_tags.py
drwxr-xr-x   8 aaugustin  staff    272 12 déc 19:47 loaders/
-rw-r--r--   1 aaugustin  staff   6205 12 déc 19:47 response.py
-rw-r--r--   1 aaugustin  staff   6245 12 déc 19:47 smartif.py
aaugustin@aaugustin Desktop % rm -rf template                                                                                        ~/Desktop
aaugustin@aaugustin Desktop % svn export https://code.djangoproject.com/svn/django/trunk/django/template                             ~/Desktop
(... snip ...)
Exported revision 17200.
aaugustin@aaugustin Desktop % ls -al template                                                                                        ~/Desktop
total 360
drwxr-xr-x  13 aaugustin  staff    442 12 déc 19:47 ./
drwx------+ 14 aaugustin  staff    476 12 déc 19:47 ../
-rw-r--r--   1 aaugustin  staff   3247 27 nov  2010 __init__.py
-rw-r--r--   1 aaugustin  staff  49210  9 déc 23:10 base.py
-rw-r--r--   1 aaugustin  staff   6146 18 nov 14:01 context.py
-rw-r--r--   1 aaugustin  staff   3586 18 nov 16:00 debug.py
-rw-r--r--   1 aaugustin  staff  28093 18 nov 14:01 defaultfilters.py
-rw-r--r--   1 aaugustin  staff  46886  9 déc 23:13 defaulttags.py
-rw-r--r--   1 aaugustin  staff   7886 21 sep 16:20 loader.py
-rw-r--r--   1 aaugustin  staff  10519  9 déc 23:10 loader_tags.py
drwxr-xr-x   7 aaugustin  staff    238 12 déc 19:47 loaders/
-rw-r--r--   1 aaugustin  staff   6205 30 sep 13:46 response.py
-rw-r--r--   1 aaugustin  staff   6245 13 jul 11:35 smartif.py
aaugustin@aaugustin Desktop % rm -rf template                                                                                        ~/Desktop

comment:6 by Aymeric Augustin, 12 years ago

Triage Stage: AcceptedReady for checkin

Marking RFC since this ticket is only of interest to core devs, there's nothing more the community can do.

comment:7 by Luke Plant, 12 years ago

Severity: NormalRelease blocker

I'm marking this a release blocker, since it is quite possibly security related: you could think you have installed a fix, but haven't at all.

comment:8 by Aymeric Augustin, 12 years ago

Unfortunately, we can only fix it in the release process, which (to the best of my knowledge) isn't documented, except in the release manager's brains...

comment:9 by Jannis Leidel, 12 years ago

Component: Core (Other)Documentation

FWIW, this doesn't happen if you use pip to install Django, so I suppose we should do two things:

  • touch the files before packaging them to reset the date
  • recommend using pip to install the Django source distribution

comment:10 by Jannis Leidel, 12 years ago

Triage Stage: Ready for checkinAccepted

by Christopher Medrela, 12 years ago

Attachment: ticket_9112.diff added

Recommending to use pip - draft.

comment:11 by Christopher Medrela, 12 years ago

Cc: krzysiumed@… added
Has patch: set

I've sent initial draft.

Some changes on download page are required.

comment:12 by Aymeric Augustin, 12 years ago

Recommending to use pip rather than easy_install to install Django (and other libs) is already covered by #9995 (#16471 being a duplicate).

Both #9995 and the docs part of this ticket should be fixed in the same commit.

comment:13 by Aymeric Augustin, 12 years ago

Owner: changed from nobody to Aymeric Augustin

comment:14 by Aymeric Augustin, 12 years ago

To clarify: my previous comment means that this ticket is only about "touch the files before packaging them to reset the date".

comment:15 by Aymeric Augustin, 12 years ago

In [17636]:

(The changeset message doesn't reference this ticket)

comment:16 by Aymeric Augustin, 12 years ago

Owner: changed from Aymeric Augustin to packagers
Triage Stage: AcceptedReady for checkin

comment:17 by Claude Paroz, 12 years ago

Is any action still required to close this ticket?

comment:18 by Jannis Leidel, 12 years ago

Resolution: fixed
Status: newclosed

Not from what I can see.

comment:19 by Harry Percival, 11 years ago

This warning still pops up when a user installs Django into a virtualenv setup with --system-site-packages, but I think it's inaccurate in that case?

http://virtualenvdemo.pythonanywhere.com/#note

Note: See TracTickets for help on using tickets.
Back to Top