Add IntervalField to database models

I'm tentatively marking this as wontfix, as surely a combination of a "start-interval" and "end-interval" fields is better normalisation.

Feel free to correct me if I'm wrong though!

start-interval and end-interval doesn't work for all situations. Suppose you want to track "Amount of time spent driving in the past week" Unless I'm misunderstanding your use of start-interval and end-interval, there isn't an easy way to store this value with date/time fields. The real bonus of intervals comes in when you are able to do computations with them (such as "Amount of time spent driving in May" + "Amount of time spent driving in June")

patches are welcome.

DurationField

This simple patch adds a DurationField, which, in my opinion, better encapsulates the use cases described by anonymous above. Calling it a duration implies that it's not tied to any specific points in time, but rather just the passage of a certain amount of time. This makes more sense for things like the length (duration!) of audio and video content, for instance.

The patch itself is fairly simple, based on a FloatField, whose value represents the number of seconds to be represented. This seems sufficient for database storage, since all supported backends already support the equivalent of a FloatField, while max_digits and decimal_places allow for a number large enough to represent timedelta.max, complete with a full set of microseconds. It doesn't take calendars into account at all, but neither does timedelta, which is also in keeping with the duration concept as opposed to intervals.

The one problem I have with it so far is that it seems like Django is relying on backend database connection modules to handle coercion into Python types, rather than using to_python, so the value isn't changed into a timedelta unless you manually call .validate() on the model. This also prevents the validation errors from occurring properly when editing in the admin. That may be another issue for another day, however. I've marked the issue as "Patch needs improvement" in case I'm missing something that would make this happen.

P.S. Pardon the WikiFormatting on the attachment comment. That is, unless it gets a wiki page, in which case it'll be fine.

I should also add, under the heading of "Patch needs improvement," that typing in decimals as a float value is hardly intuitive, and a better UI should be used. I, unfortunately, haven't thought of a good way to visually represent it, so I don't have any suggestions.

Updated DurationField location to actually work (I hadn't tested the other one)

Complete patch with widget, supporting oldforms and newforms, including newforms-admin

This most recent patch covers many more aspects of DurationField, including a newforms field and widget, fully compatible with newforms-admin. This new widget implements a bit better of a UI, separating the value into its components (days, hours, minutes, seconds, microseconds).

This new widget can't reasonably be done with oldforms, and since it's going out anyway, oldforms (and the current admin) will simply use a FloatField describing the number of seconds (including microseconds).

Also, this version of the patch relies on #3982 to enable support of the coerce method.

New patch, complete with documentation

Once again, this patch supercedes all others, and once again relies on a new patch from #3982. However, it now uses lazy instantiation of timedelta objects, which should improve performance. Documentation is also included for trunk, with a few minor changes needed for the newforms-admin branch (due to the newforms widget included in the patch). Tests will come shortly, once I have a few other patches ironed out.

Updated to work with latest trunk, and to no longer rely on #3982

This latest patch is missing a few things before it can be considered final, but it's getting close. It no longer relies on #3982, but it doesn't yet support the recent error_messages stuff, and fails with the JSON serializer. It works with the XML serializer though, and should work in newforms-admin as is.

Fixed a problem with creating objects by hand, and removed a couple debug lines

I've had some cases (mainly in the admin-interface) where get_db_prep_save(self, value) gets called with a value of type Decimal. I don't know why that happens, but I added this code to fix DurationField in these circumstances:

def get_db_prep_save(self, value):
    if value is None:
        return None
    if (type(value) == decimal.Decimal) or (type(value)==float):
        return str(value)
    return str(value.days * 24 * 3600 + value.seconds + float(value.microseconds) / 1000000)

If this is the correct way to fix this, please add it to the next diff you make.

I'm more interested to know why it's getting a Decimal, because that would probably break other expectations as well. Do you have any code that reliably shows that to happen? If you don't know exactly where it's going wrong, that's fine, just being able to reproduce it will help me figure it out.

OK... I did some testing and was able to reproduce the condition. In a simple Model with a DurationField create a record in Admin and enter 0 as the value. django-admin will throw a AttributeError?: 'Decimal' object has no attribute 'days' and will prevent saving the value.
If you enter 0 directly into the database or save a datetime.timedelta(0) from the django shell (which works, unlike in the admin) that attribute will return a Decimal("0") instead of the timedelta object (once you re-fetch it from the database).
So my proposed patch to get_db_prep_save is the wrong place to fix this problem. It seems this condition is rooted where the data is converted from to database to the native python object.
Could this have something to do with the backend driver doing the coercion to python types that was reported in #3982?
I've been testing this with sqlite3 btw.

In DurationProxy.__get__ the value that instance.__dict__[self.field_name] returns is a Decimal object (if the value in the database is 0). With other values it is a TimeDelta

This needs some UI help, but it's otherwise perfect.

Is there any plan to use PostgreSQL’s Interval type with this?

I have no plans to support interval, since DecimalField allows for compatibility with all of Django's backends. The only real reason to support interval is for inspecting existing databases that use it. If you'd like support for that, feel free to add in whatever code it would take to do it. I'm just too bogged down with other things at the moment, and supporting interval isn't high on my priority list.

I tried to apply this patch and it no longer applies cleanly. I believe django/newforms/fields.py and django/newforms/widgets.py had a reject and some fuzz.

When I did get it applied I was getting some errors when I went into the admin and tried to add data to my model...

    Traceback:
    File ".../django_trunk/django/contrib/admin/views/main.py" in add_stage
      287.         new_data = manipulator.flatten_data()
    Exception Value: 'str' object has no attribute 'items'

Related: http://www.djangosnippets.org/snippets/1060/

The 3 Best Abdominal Exercises that Are Not Traditional Ab Exercises

Updated patch for current trunk, without docs (no idea how much they've changed)

What's holding DurationField from going upstream (at least in contrib)? I can help fix whatever is necessary.

Well, you'll notice that yours is the first patch to hit this ticket since Jacob said it "needs some UI help." That criticism still stands, since a half-dozen input fields don't make for a very good user experience.

I had a look through the patch today. Using a DecimalField? and playing with commas seems unreasonably hacky. Is using a 64-bit int an option? It shouldn't be a problem under postgres, but I don't know about the other db backends.

For the interface part, there's two decent choices.
Either go for a parse of the values in a single field, for example (input) "3d 12h 4m 7s", "7d 10m", "150000 ms"; output would be highest value possible first - 60000ms would be "1m", 75000ms would be "1m 15s" (abbreviations are up for change, would go for the standard ones).

Or, go for a single field with a <select> box, with some js shortcuts. [_] [ Unit ... ] (X)

In both cases, the possible units would be years, months, weeks, days, hours, minutes, seconds, milliseconds, microseconds (assuming microsecond precision). Thoughts?

I'm a bit confused as to what you mean by "playing with commas," but thus far, I don't see anything at all hacky with using a DecimalField. Going with a 64-bit int would only be an option if Django supports a standalone cross-database 64-bit integer field. I don't want to invent one just for this. As for being hacky, I find it extremely elegant that the current approach stores seconds in the database, allowing for super-simple conversion: value = datetime.timedelta(seconds=float(value)). Plus, it works with all existing backends and maintains the ability to order records as you would expect (though, admittedly, your 64-bit int would also preserve that feature). What's exactly is hacky about it?

On the interface front, there was (once upon a time) a prevailing notion of using two text boxes: one for days and one for everything else. Basically, the days would be cleaned as a standard integer field, while the other would be cleaned as a time field, so the syntax for stuff less than a day would follow the "hh:mm:ss" format. The only thing that held me back from actually implementing that was that it would also be great to have a JavaScript helper similar to that of TimeField, showing a few useful options to get you started. Something like: 30 seconds, 2 hours, 1 day, 6 weeks (or some such similar options).

I obviously haven't gotten back around to this ticket in quite some time, but I'll see if I can devote some time to it soon to get the original UI proposal up and running.

It feels wrong to "store" as seconds. Then again, just my opinion.

I was thinking the same thing about a js helper, and a friend came up with the idea of just parsing the field for units. I'll see if I can get a few mins to work on it tonight, I really like the idea. Three big advantages:
- No need for javascript (6 weeks would be "6w", 30 seconds would be "30s", 2 hours would be "2h", etc)
- Easy to understand, very user friendly (parsing "3h 10d" is okay, I suppose we'd give validationerror on "1h 10m 2h" or similar tho)
- No need for multiple input boxes, avoids confusion

On a slightly-related note, I don't know what to do about microseconds. Standard display is µs, but that could cause problems for users with special keyboard layouts etc.

I think the year and month units are misleading, as in real life they have no fixed length. This is probably why the timedelta type does accept day as the longest unit possible.

We can assume that year is an astronomical unit, but it's almost never useful in real life. A year has to round to days and it's length is known only when applied to a certain start date. The same (or even worse) happens with a month. In Gregorian calendar there are 4 different lengths possible: 28, 29, 30, 31 days. We could assume that month is astronomical unit again (moon cycle) but it will roughly match only the Muslim calendar.

I'd prefer to leave a week as the longest unit possible.

Emes, I agree with you. I just wanted to get the implementation in. It's easier to remove than to add-on.

I sent the Dev list a mail a couple of days ago: http://groups.google.com/group/django-developers/browse_thread/thread/612e6c641d109f40?hl=en

Another question I bumped into this morning: Should we allow negative values?

cc michal@salaban.info, sorry for the spam, Firefox is horrible.

New DurationField implementation (try 3)

Any feedback on the DurationField patch? I still need to figure out the admin output.

Fully working DurationField implementation with changes and fixes from Yuri Baburov

Yuri did some updates to the patch posted above; I'll post a rebase by tomorrow. This rebase also comments out support for months and years as per list discussion; since it's just two values, might as well leave them commented.

1.1's almost out if I am to trust the targeted bugs. I suppose we could start looking at DurationField implementation for 1.2? This ticket has been open long enough.

Reassigning to myself.

Ok, 1.1 is out, it's time for an update.
This ticket has been open long enough. I'm decided to get everything closed for 1.2. I'm going to upload a rebased patch today or tomorrow. Right now, there are three outstanding issues with it:
* In the admin, a durationfield appears as a Decimal's unicode in filters, and when it's equal to zero on the admin detail page.
* Validation isn't getting handled correctly in the admin - a 500 is raised rather than a ValidationError?.
* Lack of docs/tests

First issue is something I've tried to fix but came up blank. Second one should be easy enough to fix (I'll have a look before rebasing hopefully), third one I'll leave until the former two are fixed.
I'm still *very* itchy about using a DecimalField? as backend. But hey...

Updated patch using the new BigIntegerField? as backend

There, new updated patch. See: http://groups.google.com/group/django-developers/browse_thread/thread/d32316ce7964dbaf?hl=en

Unless there are bugs, I will not work on this anymore. The core devs seem to hate the idea of having a DurationField... meanwhile I'm perfectly happy keeping my core patched.

Replying to Adys:

Unless there are bugs, I will not work on this anymore. The core devs seem to hate the idea of having a DurationField...

Yes, the core devs clearly hate the idea of DurationField. That's why this ticket was marked accepted by Jacob, and is on the low prioirty feature list for v1.2.

There is a difference between "don't like an idea" and "haven't had time to apply a patch because we've been busy with one or two other things". If we didn't like the idea, we'd close the ticket.

meanwhile I'm perfectly happy keeping my core patched.

I'll also point out that you don't need to patch core to write your own field definition. You can define fields in your own code base, and import/use them as you wish.

Adys, I liked the DurationField idea and I hope You were not seorious saying that You're going to leave it? I've tried it in my my project - there are some issues with it which I'm going to address soon - but all in all it works OK and I will be happy to help maintain it.

It would be great to see it in django itself in the future. Up to this time, I must agree with russellm that it would be easier to use/test it 'externally'. Right now, I had to create another django version (along side with my default system one from Gentoo) and patch it to use DurationField... I want to stay in-sync with Your code - that's why I didn't extract it.

Please, consider such an extraction, I belive it would made the DurationField more attractive (while it's not in django).

We are now passed the 1.2 feature freeze.