Commit graph

2832 commits

Author SHA1 Message Date
j
89311c14b4 typo 2016-08-31 01:30:13 +02:00
j
cf3161d4bd better findMedia 2016-08-31 01:30:13 +02:00
j
fc6cc93d26 keep import url 2016-08-31 01:30:13 +02:00
j
4d9e8a788a add task for queued downloads 2016-08-31 01:30:13 +02:00
j
1ec69b9c05 more py3 fixes 2016-08-31 00:33:08 +02:00
j
6aabe8d9df py3 fixes 2016-08-31 00:31:38 +02:00
j
9b3547f9bc fix tv 2016-08-28 15:05:52 +02:00
j
78c26daaeb update poster after processing stream 2016-08-28 13:38:46 +02:00
j
f71334ae25 avoid ad 2016-08-28 13:25:26 +02:00
j
da89a9748a include download task 2016-08-28 12:22:31 +02:00
j
029bcfee29 files are queued after download 2016-08-28 12:22:08 +02:00
j
11e36b550e update task after updating item 2016-08-25 18:34:49 +02:00
j
1dd574cb9e task queue has status not state 2016-08-25 18:34:49 +02:00
j
8136a2c2f5 create new item if unregistered file gets uploaded 2016-08-25 18:34:49 +02:00
j
181f27b93a fix audio only uploads 2016-08-25 18:34:49 +02:00
j
1468ddbecb towards supporting python 2 and 3
- use absolute_imports
- make use of six.moves
- use exec instead of execfile
- use list(dict) instead if dict.keys()
2016-08-25 18:34:36 +02:00
j
479432f01a upload_date can be none 2016-08-25 18:32:28 +02:00
j
83c2e25aac support editable as result in find 2016-08-25 18:32:28 +02:00
j
97a46dceb0 fix sort of static edits 2016-08-23 13:24:27 +02:00
j
9685c7e4a5 fix streams for static edits 2016-08-23 13:24:22 +02:00
j
b4074303ba avoid error in error handler and render 500 page 2016-08-23 01:37:38 +02:00
j
d7b53aa322 limit cronjob rate if multiple cronjobs end up in queue 2016-08-18 13:41:24 +02:00
j
e1cacdb67a add task queue api 2016-08-17 14:37:59 +02:00
j
0eb873d2cb fix hue, saturation, lightness calculation for multi part items 2016-08-15 15:14:44 +02:00
j
3d6d1be2e6 include oshash in video url to avoid cache issues 2016-08-13 15:36:00 +02:00
j
843cea063d update timeline if part is changed 2016-08-13 15:34:28 +02:00
j
3b8dd55273 don't fail if 2 paths get flipped in a folder 2016-08-13 15:32:51 +02:00
j
8c928860e1 import url celanup
- only import first url
- pass empty string instead of undefined
- page can contain multiple videos, import as parts
2016-08-10 10:49:58 +02:00
j
b59a9c3122 set canImportItems in config.*.jsonc 2016-08-09 15:58:48 +02:00
j
2157b719ec use more info from sort table, fixes #2953 2016-08-09 15:06:44 +02:00
j
c08ce51db2 add padma importMetadata 2016-08-09 14:25:39 +02:00
j
384c643326 no tags == [] 2016-08-09 14:25:39 +02:00
j
9991767d01 no summary in default importMetadata keys 2016-08-09 13:03:21 +01:00
j
7f8b5448e6 set default config importMetadata 2016-08-09 12:57:43 +01:00
j
0430a0f236 cache getMediaUrlInfo 2016-08-09 12:57:43 +01:00
rlx
90486ec774 update menu and config 2016-08-09 13:35:47 +02:00
rlx
5ef6062613 import media dialog 2016-08-09 13:17:48 +02:00
j
926d3f0f09 add video id 2016-08-09 11:50:24 +01:00
j
337cafcf76 don't add base keys to Item.data 2016-08-08 15:54:52 +02:00
j
9a86ddce01 include upload_date 2016-08-05 17:57:40 +02:00
j
dce51305ce update poster after extracting stream 2016-08-05 15:24:54 +02:00
j
c4a95bc320 update_poster should always update poster 2016-08-05 14:46:27 +02:00
j
2aef460225 do not set -ac if audiochannels is None 2016-08-05 14:46:27 +02:00
j
ff8dd16456 fix profile selection 2016-08-03 11:41:46 +02:00
j
5447aab0a1 more current values 2016-07-30 12:04:35 +02:00
j
dcf3acbd44 store facets in NFKD 2016-07-30 12:01:34 +02:00
j
dffb25e0e8 poster might still point to old id, after rename, delete and make new one. fixes #2944 2016-07-30 10:02:53 +02:00
j
5c90ff79a0 cleanup poster cache 2016-07-30 09:39:31 +02:00
j
893f2e5423 add importMediaDialog 2016-07-30 02:49:43 +02:00
j
383ad8a535 fix findMedia by item id 2016-07-24 17:52:33 +02:00
j
6f4a0ba90a support vtt subtitles 2016-07-01 21:13:13 +02:00
j
f8f951c966 remove outdated fixmes 2016-07-01 17:53:59 +02:00
j
d780045306 flake8 2016-07-01 17:48:43 +02:00
j
3166bf5beb show timeline errors 2016-07-01 17:34:45 +02:00
j
5434ffbf53 make_poster no longer takes arguments 2016-07-01 12:44:07 +02:00
j
88c800a093 use data['positions'] 2016-06-30 16:42:56 +02:00
j
c7157d5001 only decode ids for found positions
merge document.view.get_positions into item.utils.get_positions
add flag to decode id before looking up in ids

Followup to 09ebbc9cc6
2016-06-30 16:18:07 +02:00
j
3d95d7013a Merge remote-tracking branch 'wjt/2935-findDocuments-performance' 2016-06-30 15:43:09 +02:00
j
e9863c238e always use get_operator, avoid case-insensitive match if possible 2016-06-30 11:28:09 +02:00
j
5b545e6e43 layer first 2016-06-30 02:42:24 +02:00
j
b6e4d33a43 add Annotation.findvalue index (used by findClips) 2016-06-30 02:42:24 +02:00
j
b63024df02 show source info if encode fails 2016-06-30 00:23:56 +02:00
j
b5bc17f925 make layer itemKeys with filter=true show up in searching for * 2016-06-30 00:17:23 +02:00
09ebbc9cc6
findDocuments: improve performance of positions queries
References #2935
2016-06-28 16:36:55 +01:00
8d25e3be78
findDocuments: improve entity query performance
When I implemented this in 9a4c24c, there were not many rows in
entity_documentproperties in the database here. Now that there are,
computing the document_document -> entity_documentproperties ->
entity_entity join and then filtering is really, really slow. Postgres
seems to materialize the whole join and then scan it.

If we get a set of matching document IDs for the entity query in a
subquery, and then just filter with IN on that, things are much faster:
scan entity_entity; in a nested loop, get the document_ids via
entity_documentproperties; hash this set; and then scan
document_document.

Searching for a single character, this brings the query from ~1.1s to
~400ms. Searching for a full word, ~800ms to 120ms

This condition is getting really ugly -- I am sorry!

References #2935
2016-06-28 16:33:01 +01:00
j
5aeffcfb6a check first audio track 2016-06-27 16:51:18 +02:00
j
adfcc1cb27 never set display aspect ratio to 0:0 2016-06-27 16:08:30 +02:00
j
8ac78f3bd6 remove unused force flag from make_poster, update_timeline 2016-06-26 23:24:11 +02:00
j
0f9e80e1e6 avoid saving item twice 2016-06-26 23:22:27 +02:00
j
de9b062d63 make sure existing index is using gin 2016-06-26 16:55:58 +02:00
j
ab0dfddf31 set SECURE_PROXY_SSL_HEADER by default 2016-06-26 15:34:19 +02:00
j
0d89ad640b ignore some broken audio codecs 2016-06-26 15:33:52 +02:00
j
92f642cbac pcm sound can have no codec 2016-06-26 14:41:58 +02:00
j
2cec1b9ad5 s/import Image/from PIL import Image/g 2016-06-25 20:39:29 +02:00
j
4785f314cb Add VP9/Opus support, use VP8 by default
- support vp9 and opus
- switch to 2 pass encoding
- use ffmpeg -movflags +faststart instead of qtfaststart
2016-06-23 17:36:41 +02:00
j
aaacc48259 only save if update_external fails 2016-06-20 18:28:05 +02:00
j
d83647c4a5 don't hide oxtimelines errors 2016-06-20 18:27:31 +02:00
j
6dcbcdd19c dont update timeline in update_selected, remove unused async get_item case 2016-06-16 14:48:54 +02:00
j
0486d62ec9 use absolute path 2016-06-16 14:48:09 +02:00
j
f25218466b formating 2016-06-16 14:48:01 +02:00
j
70f34bfde9 typo 2016-06-15 19:13:00 +02:00
j
e3c5ab18c7 only update itemsort if name is changed 2016-06-15 18:31:40 +02:00
j
22f83288c5 avoid looking up item twice 2016-06-15 18:29:09 +02:00
j
7c53dca65b less async item creation 2016-06-15 18:12:59 +02:00
j
b2a9a5f711 space 2016-06-15 17:56:31 +02:00
j
3c1f4a8c95 dont call module 2016-06-15 17:55:57 +02:00
j
b010aca0a9 s/taskId/id/ 2016-06-15 15:45:51 +02:00
j
a0fc6ffadc typo 2016-06-15 14:55:45 +02:00
j
f4cbe6a114 return empty sequences if no data timeline exists 2016-06-15 14:48:02 +02:00
j
af0e0cffe8 person can be removed again, let async itemsort fail without exception 2016-06-15 14:34:46 +02:00
j
fd9d3bdabf flake8 + map->[] 2016-06-15 14:34:46 +02:00
j
05c4cfcbc8 add space and other flake8 cleanups 2016-05-28 11:30:43 +02:00
j
5e149a5cb8 add space and other flake8 cleanups 2016-05-28 11:26:46 +02:00
j
225259e521 add space and other flake8 cleanups 2016-05-28 11:18:51 +02:00
j
f21e8413fb use get_random_string 2016-05-28 11:18:51 +02:00
j
7fdaf6d1ce include Access-Control-Allow-Origin in 404 not found response 2016-05-27 11:51:47 +02:00
05e6118a88
findAnnotations: include duration alongside result count
fixes #2921
2016-05-05 15:54:25 +01:00
j
41cc8e3573 expose encoding status via api 2016-05-05 10:49:34 +02:00
j
be163826ef Merge remote-tracking branch 'wjt/fix-migrations' 2016-05-05 10:48:24 +02:00
39b9b48be2
archive: fix migrations for upload_to function renamings
9c75526 renamed these functions. The function doesn't affect the DB
schema so it should be safe to just modify the migraiton.
2016-05-04 17:01:44 +01:00
e29ea230fb
Add migration for Document.documentproperties ref
This should have been included with a8dcbbb, which changed the
related_name to access DocumentProperties from Document. (There's no
actual change to the database.)
2016-05-04 16:55:11 +01:00
j
0f28a2b7d5 fix queue status 2016-04-30 14:15:13 +02:00
j
9c7552699f fix upload_to callbacks 2016-04-29 13:46:55 +02:00
2812834ce3
findAnnotations: don't lowercase ids (fixes #2916)
Without this fix, a condition like:

     {key: 'id', operator: '==', value: 'A/B'}

gets mapped to:

     public_id__exact=('A/B'.lower())

which is wrong.

I introduced this bug in b3df5b8. I didn't catch it because I was
mostly interested in the 'layer' key -- but layer names are
conventionally lowercase anyway so lowercasing them had no effect.
2016-04-29 11:03:45 +01:00
aa40a40595
Annotation.json: only include entity id & name
Fetching documents for each entity in turn is expensive. (I have tried
using ArrayAgg to fetch them in the same query as the Entity — no
improvement. It's possible that being able to join to entity_entity,
and then use ArrayAgg, would be better.)

Even once you've fetched them all, if the same entity appears many
times in an item, then get(..., keys=['layers']) duplicates the whole
JSON for the entity many times: expensive to serialize, expensive to
send over the wire.

Pandora's own web interface only depends on the 'id' key of 'entity' in
each annotation, and refetches the rest of the entity to show the pop-up
dialog when you press E. So by just not bothering to fetch and send any
other keys, get(..., keys=['layers']) on an item with many entity
annotations is substantially faster.

(I experimented with splitting the full entities off to one side, so,
you'd have:

    {
        "layers": {
            somelayer: [...,
              {..., "entity": {"id": ABC}},
            ], ...
        },
        "entities": {
            ABC: {...},
            ...
        }
    }

This is quicker than the status quo, but obviously not as fast as not
fetching & sending the rest at all!)
2016-04-28 14:15:23 +01:00
aa0fbc9d4a
Entity.json: get document ids from join table
This is a bit quicker because it's just a lookup in a single table, not
a join.
2016-04-28 14:15:12 +01:00
400b6650a2
Annotation.json: document empty-subtitle special case 2016-04-19 13:52:52 +01:00
af0d87b569
Annotation.json: reduce repeated layer lookups
It's actually quite costly to look up keys in CONFIG, particularly
inside a loop: this trims ~5% off get(keys=['layers']) for
annotation-heavy items.
2016-04-19 13:52:47 +01:00
3f5be0bd27
findClips: look up entity names (fixes #2804) 2016-04-19 12:28:58 +01:00
d0129a4416
findClips: avoid O(n²) lookup of clip from annotation
This doesn't make much difference for small ranges, of course.
2016-04-19 11:25:12 +01:00
ba00bcbf7b
findClips: select_related('item') / ('item__sort')
Clip.public_id uses self.item.public_id.

Clip.json() uses self.item.sort, so we should select_related on that
rather than the clip's own sort field. (They are identical objects. Is
Clip.sort ever used directly?)

With this change, findClips() issues one query to fetch clips plus one
query per flavour of annotation; before, it issued two extra queries per
clip.
2016-04-19 11:25:06 +01:00
6dbb7f921a
findClips: only scan layers once 2016-04-19 11:14:25 +01:00
b3df5b8d56 findAnnotations: match some fields case-sensitively
Requiring layer to have the right case is consistent with
addAnnotation(), and means the _layer[_like] index can be used. In my
testing, if itemsQuery specifies a single item, then postgres doesn't
bother with the layer index anyway; but if not, it makes a pretty big
(~3×) difference.

Matching public_id and item__public_id case-sensitively also seems
reasonable (it's consistent with get() and getAnnotation()).

(Is lower() redundant for the case-insensitive comparisons? ie. is
UPPER(x.lower()) == UPPER(x)? I'm not sure, it's cheap, let's leave it.)
2016-04-05 12:19:32 +01:00
8d1b4de337 findAnnotations(): make 'findvalue' the default key
Annotations have no 'name' field, so

     findAnnotations({query: {conditions: [{value: 'foo'}]}})

would previously raise an exception.
2016-04-05 12:19:31 +01:00
284caf03c3 get_by_key: short-circuit
This is about 30% faster, presumably because it avoids allocation and/or
closing over variables is slow(?). It's not hugely significant (I
misread a line_profile report) but why not.
2016-04-05 12:19:31 +01:00
j
7ac68697d4 update pdf.js 2016-04-04 15:50:07 +02:00
j
652df88342 return 404 2016-04-04 15:50:07 +02:00
j
1bff4aa0e9 avoid storing invalid poster frames, only show videos with video 2016-04-01 16:40:20 +02:00
j
b8beb51480 fix multipart audio only timelines 2016-03-31 14:54:38 +02:00
j
94b940436f fix timelines for items with many parts
- use durations from streams not from timelines
 - don't accumulate timeline drift
2016-03-19 18:58:48 +01:00
j
f0b8b2b81e check that range is [int, int] 2016-03-17 16:06:08 +01:00
eeaeda3970 Support WebVTT subtitle export 2016-03-11 14:16:23 +01:00
j
697e501a4f only update item timeline once all parts are done 2016-03-11 10:33:48 +01:00
j
f6cebcaec9 fix user/group api 2016-03-08 20:14:05 +05:30
j
29204b6fb5 move gunicorn configuration from init script to config file 2016-03-07 14:25:24 +05:30
j
4f28c2c548 fix annotation import, values are decoded in d1.9 2016-03-05 15:36:47 +05:30
a8dcbbbe89 Include DocumentProperties.data in Document.json() 2016-03-05 15:07:47 +05:30
a55cbcfb9f DocumentProperties: add data field 2016-03-05 15:07:47 +05:30
0c98cd080e Entity.alternativeNames: default to () not [] (fixes #2896)
Otherwise this:

    self.name_find = '||' + '||'.join((self.name,) + self.alternativeNames) + '||'
                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

fails because () + [] is an error. I guess this must have been
introduced by the DictField/TupleField rewrite.

Without this fix, it is impossible to create a new entity.

Basically the same logic is used for Event and Place too so I've made
the same change to those, and, in passing, fix another copy of the bug
fixed for Entity.name_find in fe7f961.
2016-03-04 17:11:36 +00:00
9a4c24cdb4 Support searching documents by entities 2016-03-04 12:41:41 +00:00
738a9282b4 Document: fix negating id queries 2016-03-04 12:41:41 +00:00
8c23bdff6d Implement DocumentProperties.__unicode__ 2016-03-04 12:41:41 +00:00
j
4613005b83 use geoip2 api to fix ipv6 lookups 2016-03-04 12:50:44 +05:30
c6f9f87c8e Fix autocompleteSort with multiple keys (fixes #2893)
QuerySet.order_by() takes each key as a separate argument, not as a
single comma-separated string.
2016-03-03 18:15:37 +05:30
2a07e2a1ab Remove redundant overrides of Model.delete
Both of these models have pre_delete handlers which do the same things,
so I think these are unnecessary.
2016-03-03 18:10:29 +05:30
d69a8efd97 Don't save other file-owning models on delete, either 2016-03-03 18:10:29 +05:30
6e0049a20c Don't save Document in pre_delete handler (fixes #2889)
FileField.delete() will, by default, save() the model instance it is
attached to. This is pointless if we're in the process of deleting the
Document -- and since Document.save() calls Document.update_matches(),
this scans all annotations every time a document is deleted.
2016-03-03 18:10:29 +05:30
j
fd2c69a5b7 define BASE_DIR 2016-03-03 16:56:06 +05:30
j
11924fcb0d Rename api.taskStatus to api.getTaskStatus
still register api.taskStatus since its used
in pandora_client and other scripts.

fixes #2600 and fixes #2859
2016-02-20 18:35:26 +05:30
j
0d5c968026 space 2016-02-20 10:41:23 +00:00
j
051a3578b4 inline django 1.9 version of ox.django 2016-02-20 10:41:23 +00:00
j
7620795862 add update_itemsfolder command 2016-02-20 10:41:22 +00:00
j
0367816257 only crop with 4 crop points 2016-02-20 10:41:22 +00:00
j
1ef547f885 update celery integration 2016-02-20 10:41:22 +00:00
j
f060601d40 django 1.9 returns parsed value not raw data 2016-02-20 15:15:02 +05:30
j
e9bcd19161 fix initdb 2016-02-20 15:15:02 +05:30
j
29c6dc866b fix index 2016-02-20 15:15:02 +05:30
j
7a6515183a add wsgi 2016-02-20 15:15:02 +05:30
j
cc84487193 commit_unless_managed->commit 2016-02-20 15:15:02 +05:30
j
62caf55fe6 stop using RequestContext 2016-02-20 15:15:02 +05:30
j
3f02f6086a fix clip random 2016-02-20 15:15:01 +05:30
j
2fe857d4fd remove optparse 2016-02-20 15:15:01 +05:30
j
4219246909 db migration 2016-02-20 15:15:01 +05:30
j
976860ec30 use django.conf.urls url instead of patterns 2016-02-20 06:00:13 +00:00
j
158c68fbc7 use object class for MetaClip 2016-02-20 05:58:53 +00:00
j
1cc795995d user.get_profile()->user.profile 2016-02-19 22:06:11 +05:30
j
3e8f28ff47 update imports 2016-02-19 22:06:10 +05:30
j
5d63ad1760 default/upload_to=lambda breaks migrations, use functions 2016-02-19 21:59:26 +05:30
j
e22cc432b3 use transaction.atomic 2016-02-19 21:55:09 +05:30
j
e738503380 remove old migrations 2016-02-19 15:17:18 +00:00
j
35bdfba77c fix default values 2016-02-19 19:01:38 +05:30
j
d241b90b63 check data is a dict 2016-02-19 18:46:53 +05:30
j
6b1ce30eb2 from __future__ import print_function 2016-02-18 16:19:26 +05:30
47d8d67c3d Allow 'query' to be omitted from find*() (fixes #2883)
I hit this in findDocuments() so thought I'd fix them all.
2016-02-15 14:58:50 +00:00
j
b121b58a86 support caching local files, lookup by oshash 2016-02-14 00:32:14 +05:30
691b3fa934 Entity: only update annotations when name has changed (fixes #2877)
Just like ace0468 this doesn't exactly fix #2827, but it does reduce the
unnecessary work.
2016-02-05 19:03:16 +05:30
6008f3b44e findEntities: fix negated conditions, fixes #2876 2016-02-05 19:01:30 +05:30
cdaa50c4d3 addAnnotations: report which key is missing in error 2016-02-04 10:38:02 +00:00
729856eb30 Rename getTaskStatus -> taskStatus in API docs (fixes #2859) 2016-02-04 10:38:01 +00:00
j
c7515517da remove persona support 2016-01-12 14:04:34 +05:30
j
face7da3da avoid duplicate users for or queries, fixes #2871 2016-01-12 12:06:12 +05:30
j
f47febb09c joined audio timelines 2015-12-09 18:29:18 +00:00
j
d23a6eb317 fix cuts for multipart videos 2015-12-01 20:11:08 +01:00
j
a919724821 typo 2015-11-29 14:13:44 +01:00
j
f6bfa26ae3 dont count empty annots 2015-11-26 09:21:52 +01:00
j
8f6f39caaf dont include emtpy annotations in srt export 2015-11-20 18:19:55 +01:00
j
2b3853c7aa remove debug 2015-11-18 14:45:31 +01:00
j
e216a7b307 make it easer to copy&paste default values 2015-11-18 14:28:40 +01:00
j
48e1435601 switch to git 2015-11-14 15:22:40 +01:00
j
b7cfcfa182 add some encoding options: crop, trim, aspect 2015-11-12 16:15:02 +00:00
j
310343c81c improve output if config can not be parsed 2015-11-12 13:06:34 +01:00
j
9e88f29126 only loop over annotations without public_id once 2015-11-11 10:39:34 +01:00
j
9b1ed58ff2 only migrate AnnotationSequence if we have items 2015-11-11 09:48:16 +01:00
j
4bdce55364 fix joining audio only timelines 2015-11-02 09:34:41 +01:00
j
eb472b88ae allow annotation import/export on 0x 2015-10-31 13:54:12 +01:00
j
54b7c4be9a update dimensions replacing documents, add modified to image urls to get latest version 2015-10-26 16:10:12 +01:00
j
6f9fb06da3 Replace Document, still some cache issues, fixes #2855 2015-10-25 22:08:13 +01:00
j
21a6007f8c only add video sitemap for films that can be played 2015-10-22 19:40:20 +02:00
j
90d8d46603 update sitemap video namespace 2015-10-22 00:11:44 +02:00
j
86599a4341 Indiancine.ma: new items should be Under Copyright by default not Private 2015-10-21 14:26:13 +02:00
j
f0e3c2775e logg addMedia call 2015-10-20 12:17:04 +02:00
j
d16bbf6ba8 remove facet duplicates 2015-10-12 18:44:20 +01:00
j
c17f0a4376 add new file for r5049 2015-10-12 18:43:41 +01:00
j
aafac3c1d8 only store one item/key/value facet, remove facets with other case 2015-10-12 17:45:08 +02:00
rolux
69254bbe48 changeid can also be list of ids, use text 2015-10-12 15:25:24 +01:00
rolux
4ed2d940cf migrate annotation sequence in item not annotation 2015-10-12 15:24:03 +01:00
j
2e3b61d163 dont fail if files are already gone 2015-10-04 18:17:34 +01:00
j
6f4c010be0 only return layers defined in config 2015-10-04 14:11:39 +01:00
j
be1589569e fix clip index for newly added clips 2015-10-04 11:04:46 +01:00
j
5649892bbd annotation layer flag is boolean 2015-10-04 11:20:45 +02:00
9265b8a53b Clip.save: fetch annotations once, not ~ 2 * n_layers
With 17 layers and 12 clipLayers, this repeated fetching was around 49%
of the cost of this function, which was in turn 94% of the cost of
creating many new annotations with mostly-unique endpoints. This helps a
bit...

If the order of clipLayers is not meant to be significant to sortvalue
(which I assume it is) then this could be simpler.
2015-10-04 11:17:22 +02:00
j
a430c6bdf4 make facets case insensitive 2015-09-25 14:44:02 +01:00
j
1e81dc4fa1 switch 2015-09-24 18:35:13 +01:00
j
79dbeabafc not again 2015-09-24 18:26:59 +01:00
j
eea9321b2e not not not 2015-09-24 18:23:25 +01:00
j
96301d6a9c not 2015-09-24 18:21:01 +01:00
j
f12dfdc4a3 import subtitles if no subtitles exist 2015-09-24 18:16:29 +01:00
j
65fb9ccb6d reduce ffmpeg output 2015-09-16 15:04:44 +01:00
4f064fda76 Make Annotation.public_id non-NULLable (fixes #2829)
This fixes this race:

     request 1                          request 2
     -----------------------------      -------------------------
     addAnnotation(...)
     super(Annotation.self).save()
                                        findAnnotations(...)
                                        returns [{id: null, ...}]
     annotation.public_id = x
     returns {id: x}
2015-09-14 14:18:10 +02:00
eaa07b1ccb ClipManager: match annotation layer case-sensitively (fixes #2832)
The case must be correct anyway for the layer to be found in
settings.CONFIG['layers']. Running this:

    Q(annotation__layer__iexact='foo') &
    Q(annotation__findvalue__icontains='bar')

compiles to

    upper(layer) = upper('foo') and
    ...

which can't use the case-sensitive annotation_annotation_layer index.
This:

    Q(annotation__layer__exact='foo') &
    Q(annotation__findvalue__icontains='bar')

can. (It still can't use the findvalue_like index, though! The other
option is to add indices on upper(layer) and upper(findvalue)
[varchar_pattern_ops].)
2015-09-14 14:13:06 +02:00
da1ad5b9c1 ClipManager.filter_annotations: fix 'opterator' typo (fixes #2832) 2015-09-14 14:11:40 +02:00
8759b569da Cache serialized entities when fetching many annotations
For a scene with ~5600 annotations, of which ~3100 are entities, this
cuts fetching the scene from 12 seconds to 2 seconds.
2015-09-14 14:08:02 +02:00
eebb0b5681 Combine {Item,Clip,edit.Clip}.get_layers()
This has several benefits:

    • Clip.get_layers() (used by smart edits) and Item.get_layers() pick up
    the select_related('user') optimization added for static edits in
    r5007.

    • Static edits and items pick up the optimization from r4941 to select
    annotations once, not once per layer.

Fetching an item with ~1000 annotations took ~1s without this patch,
~0.34s with this patch. Another item with ~6000 annotations took ~11.6s
before, ~8.6s after.

Because this block is moved out to the top:

if user and user.is_anonymous():
user = None

then, for anonymous users,

"editable": false,

is no longer included in the annotations. The old behaviour ended up
including this key in all layers listed before the first private layer
in the config, and leaving it out from later ones. So this new behaviour
is more consistent.
2015-09-14 14:06:43 +02:00
j
2c406a76e0 create trusty container 2015-09-03 19:33:48 +02:00
j
fd2992c588 add option to sort by number of annotations per layer 2015-09-03 00:52:20 +02:00
ace04688f2 Entity.save(): update annotations async (fixes #2827, kinda) 2015-09-02 14:32:16 +02:00
j
41b50ccdb8 add canPlayClips flag to annotation layers and use those layers to limit playback to clips 2015-08-27 11:27:27 +02:00
j
83013bbe5e Update items when entities are renamed (fixes #2825) 2015-08-26 19:42:03 +02:00
j
944fe1a9dd only run migration if we have items 2015-08-07 17:32:17 +02:00
3da3bd37fd addClips: return error if item/in/out missing, not 500 2015-08-07 13:37:56 +02:00
j
819181726a slightly faster json serialization of annotations 2015-08-02 16:22:45 +02:00
f3fdded07d Edit.json: preload annotation users
The expensive part of fetching an edit is JSONifying the clips'
annotations. Profiling showed that the main cost was Annotation.json(),
and within that:

File: /srv/pandora/pandora/annotation/models.py
Function: json at line 216

Line #      Hits         Time  Per Hit   % Time  Line Contents
==============================================================
216                                               def json(self, layer=False, keys=None, user=None):
217       632          827      1.3      0.1          j = {
218       632      1048170   1658.5     89.6              'user': self.user.username,
219                                                   }

Obviously this join just moves some of the cost further out, but it
brings my micro-benchmark down from 1.3s to 0.3s.
2015-08-02 16:02:47 +02:00
ab5a20d3a2 Treat findEdits({}) like findEdits({query: {}}) (fixes #2820) 2015-07-22 21:37:55 +02:00
d3c18a5859 editAnnotation: explicitly refuse to change layer, fixes #2818 2015-07-14 10:48:05 +02:00
1ef7101761 annotations: fix 'unkown' typo 2015-07-14 10:47:00 +02:00
j
5e6b054896 extend cookie age if its still used 2015-06-30 20:02:09 +02:00
j
9208e09dd2 sync config documentation 2015-06-26 07:27:56 +00:00
j
1a2f051bdd fix documentation update 2015-06-26 07:27:13 +00:00
j
9758b7d312 fail if entity is unknown 2015-06-24 16:41:49 +02:00
j
ce701d38d0 space 2015-06-24 16:41:11 +02:00
j
adb831ce70 dont turn data into string, fixes #2797 2015-05-28 13:10:39 +02:00
j
5da83f6f6d don't return empty string for undefined values, handle in context browser 2015-05-23 15:11:18 +05:30
j
0504eaf227 audio files should not have aspect ratio 2015-05-23 15:10:24 +05:30
j
9191a9c196 extension should be lowercase 2015-05-23 11:06:22 +02:00
j
27e9bc62fb typo 2015-05-23 14:25:58 +05:30
j
23398c4307 get resolution from videos while uploading and dont return sort value 2015-05-23 14:22:44 +05:30
j
7359a24d78 keep video height if its < profile height 2015-05-22 19:01:05 +05:30
8b0459592c logError: log HTTP_USER_AGENT
include user agent in error text
2015-05-20 12:38:49 +02:00
3ad9d4821b Fix findEntities( ... name == x ... ), fixes #2769 2015-05-20 12:30:31 +02:00
fe7f961ff6 Entity: fix name_find when alternativeNames is non-empty
Previously, given:

    name = "foo"
    alternativeNames = ("bar", "baz")

then:

    name_find = '||foobar||baz||'
2015-05-20 12:16:32 +02:00
b7a83554ab Fix references to findDocument[s] in docstrings 2015-05-20 12:14:21 +02:00
j
f11772b8db fix annotations without public_id after initializing sequence 2015-05-20 01:04:42 +05:30
j
ef56f4aec4 track annotation id in table to avoid IntegrityErrors while adding multiple annotations to one item, fixes #2780 2015-05-20 00:43:33 +05:30
j
34f2cfe290 avoid updating item if another annotation was added or edited since dispatching the update_item event 2015-05-20 00:20:50 +05:30
j
47a3a24a7d ignore messages that can not be serialized 2015-05-19 02:39:26 +05:30
j
af86445b2e use username not user 2015-05-18 23:01:19 +02:00
j
6dd9522cc5 never include empty layers 2015-05-13 21:59:17 +02:00
j
562a506d68 include user in change event 2015-05-08 08:39:31 +02:00
f9ba4d1bf1 findChangeLog: don't crash if sort is unspecified
This would crash:

    api.findChangeLogs(keys=[])

because there's no 'name' key for changelog entries. Instead, default to
chronological order, newest first.
2015-05-06 19:00:44 +02:00
514e1ad36d autocompleteEntities: fix lies in docstring 2015-04-30 22:49:31 +02:00
4d6466779f autocomplete: fix typo in docstring 2015-04-30 22:48:07 +02:00
j
e7f83f674e add inital implementation for a websocket, disabled by default for now 2015-04-28 23:05:15 +05:30
j
7aa609c246 make sure app.tasks gets loaded 2015-04-28 22:34:45 +05:30
c2e7ac3190 signup: fix error documentation, require email
email is required by the UI, and if you don't provide
it this method raises KeyError anyway.
2015-04-28 18:13:25 +02:00
51c3df9660 typo: permissino permission in 403 errors 2015-04-28 18:12:00 +02:00
j
d39c237b04 ignore subtitles that are longer than the video. 2015-04-25 19:18:32 +02:00
j
abf445b7f1 faster edits 2015-04-25 19:02:34 +02:00
j
e666159811 faster find queries 2015-04-25 19:02:20 +02:00
j
9197591bb6 tune smart clip queries 2015-04-25 16:13:42 +02:00
j
e604ba73bd remove debug output 2015-04-25 13:08:24 +02:00
j
8abed26dc9 make sure alternative names are unique too, also fixes #2754 2015-04-23 17:20:00 +05:30
j
5b7b389661 ignore case checking for existing entities, check for alternative names too, migrate existing double entries, fixes #2754 2015-04-23 12:55:11 +02:00
j
9d3c50fcec ignore undefined entity data 2015-04-21 15:34:37 +05:30
j
0b103afeca minify before overwriting old file; oxjs removes old tree, always build geo 2015-04-18 16:06:56 +05:30
rolux
2ad4aad1c5 fix typo in config documentation 2015-04-20 10:48:04 +02:00
rolux
442e847747 fix typo in config documentation 2015-04-20 10:47:35 +02:00
rolux
3208db2fb0 update config documentation 2015-04-20 10:46:00 +02:00
j
2dc081b8aa store name, remove unused keys 2015-04-17 16:01:49 +01:00
j
10183a9f06 only create index in sqlfindindex not during migration 2015-04-17 11:34:44 +01:00
j
9cbf62267f split data migration into extra step 2015-04-17 11:26:56 +01:00
j
e407db4605 fix sqlfindindex 2015-04-17 11:22:34 +01:00
j
a869c7a89a extend findEntities to support entity keys and * 2015-04-17 11:17:12 +01:00
j
e851dcabfb entities are not links 2015-04-16 20:02:31 +00:00
j
c59fb1596f fix entities with ' in name 2015-04-16 20:50:59 +05:30
j
1fb075a350 rename layer 2015-04-16 10:53:05 +01:00
j
ff5d9633ce add db index on changelog.created 2015-04-15 15:00:52 +05:30
j
5e8294b1c0 function to merge two users 2015-04-15 14:59:42 +05:30
j
6793c165b3 refactor filters updates 2015-04-15 08:51:34 +01:00
j
6a5280edd4 ignore case for annotatoin filter values 2015-04-14 23:04:07 +01:00
j
2325c71c21 fix limit by created 2015-04-14 20:13:19 +01:00
rolux
71fd1314cf document new layer/autocomplete config option 2015-04-14 19:59:18 +01:00
j
3e0bd078d0 pass autocomplete with layer, support autocomplete for leyers with filters, fixes #2736 2015-04-14 19:49:22 +01:00
j
c73ac10ed4 log addEntity in changelog 2015-04-14 19:15:59 +01:00
j
ae55130f47 add subtitles offset, fixes #2732 2015-04-14 15:53:47 +01:00
j
083fabc208 include first annotation as annotation id for clips 2015-04-02 13:43:46 +02:00
j
ea52722d9f show date instead of undefined in results grid, fixes #2727 2015-03-29 22:54:05 +05:30
j
253125e3c6 pad.ma does not have year 2015-03-28 14:27:55 +01:00
j
7b401cfe28 round duration to 3 2015-03-26 20:28:07 +05:30
j
bf1dde53d6 fix ignoring media files without instances 2015-03-23 11:27:34 +05:30
j
15e2f1e844 exit on error 2015-03-20 05:31:05 +00:00
9ac5dc7ba9 [PATCH] Fix find()'s docs on nested subconditions
The documentation says that you need to write (P & Q) | R
like this:

{
    'conditions': [
        {
            'query': {
                'conditions': [P, Q],
                'operator': '&'
            }
        },
        R
    ],
    'operator': '|'
}

but actually you need to omit the { 'query': ... } wrapping the
subclause. Including it crashes parseCondition() at:

    v = condition['value']

for the obvious reason.
2015-03-05 14:17:02 +05:30
j
1ed91da4e0 fix import 2015-02-26 06:07:18 +00:00
j
deda6252f1 keep entity annotation find/sort values in sync with entities 2015-02-26 06:04:16 +00:00
j
7377de62f7 remove all sessions of user on signout, fixes #2700 2015-02-25 18:17:35 +00:00
j
90a49b2a4d ui.onload -> user.script, fixes #2697 2015-02-21 10:32:48 +00:00
j
441cb9892a allow updates of file metadata for files with info 2015-02-21 15:35:01 +05:30
j
6693bfcd0f return document or 404 no 500 2015-02-20 15:13:23 +00:00
j
2a63a3b300 dont include null values in lookup 2015-02-16 16:12:44 +00:00
j
484b33e39d add option to return item keys in findAnnotation 2015-02-16 16:10:45 +00:00
j
2e246dd06e cleanup 2015-02-16 16:02:18 +00:00