Commit graph

3088 commits

Author SHA1 Message Date
j
91e0f588e5 only append to documentKeys after adding defaults 2018-06-14 19:27:40 +02:00
j
7acc562b53 only admins can change rightslevel 2018-05-31 17:27:05 +02:00
j
717248bc73 fix path name of multi track videos 2018-05-22 01:27:43 +02:00
j
c4c0a037ce typo 2018-05-09 18:47:36 +01:00
j
4cd1473a94 get file order from streams 2018-05-09 18:39:06 +01:00
j
099c2610e6 ignore unknown tasks 2018-05-09 15:12:25 +01:00
j
e8e5ce415d enable autocomplete by default 2018-05-08 12:33:39 +01:00
j
77952b7e78 no tos for now 2018-03-31 16:40:29 +05:30
j
b84b8cb6e9 only canEditEvents/canEditPlaces can add new places/events 2018-03-31 14:01:39 +05:30
j
e4288123da allow members to edit places/events 2018-03-30 13:18:42 +05:30
j
660c89bc19 use tagline 2018-03-10 11:07:34 +05:30
j
1dc5969cbc fix rename user 2018-03-09 16:03:52 +05:30
j
4dbc7930b4 find in public notes 2018-03-06 19:26:18 +05:30
j
859a820c90 add ui for lists/edits/collections groups 2018-03-06 15:11:05 +05:30
j
c32503e1af log full error url 2018-02-27 19:11:08 +05:30
j
c6ddf58cac add keywords filter to indiancine.ma config, fixes #3087 2018-02-23 16:41:00 +05:30
j
3920541bb8 sort documents by title, fall back to created 2018-02-21 12:44:35 +05:30
j
88d47eba6b don't decode str 2018-02-21 08:52:22 +05:30
j
1a3edf4f93 sort by document name 2018-02-15 15:57:12 +01:00
j
a73b9f5600 use functools directly 2018-02-15 15:57:03 +01:00
j
5ac305c620 pass user 2018-01-15 12:20:01 +01:00
j
2a0c2acc97 user can be none 2018-01-15 12:16:20 +01:00
j
046c49cd3b send no-store for temp poster 2018-01-14 20:55:50 +01:00
j
e13ef4aa1e redirect to actual url 2018-01-14 20:46:02 +01:00
j
12ad6b02fe send redirect, otherwise nginx takes expiry for static url 2018-01-14 20:42:23 +01:00
j
341e1bd892 body is bytes 2018-01-12 11:37:20 +01:00
j
f44584f131 allow cross origin content-type header 2018-01-12 11:35:37 +01:00
j
6c6b60ffaf lower auth deps 2017-12-31 14:21:18 +01:00
j
85c5fafe0d add groups to collections, edits and lists, towards #3063 2017-12-31 14:06:27 +01:00
j
15c5ad9d54 pad.ma is public 2017-12-30 13:55:59 +01:00
j
2bfaf3809b documentRightsLevel 2017-12-15 13:12:30 +00:00
j
c640d1253e dynamic /robots.txt 2017-12-15 12:59:56 +00:00
j
3b73fe78bd update user references if username gets changed 2017-11-16 18:05:50 +01:00
j
959ffaba25 use ffmpeg to merge mp4 files, support clip extraction from multiple parts 2017-11-16 16:39:38 +01:00
j
a5c6cb875f fix country lookup 2017-11-14 18:04:09 +00:00
j
53fdb52ee8 cdn prefix 2017-11-14 18:40:20 +01:00
j
d403125757 default download format 2017-11-14 17:13:41 +01:00
j
5ce9d605f7 refactor geo lookup 2017-11-14 17:06:02 +01:00
j
96bd3d6df8 don't serve debug data by default 2017-11-14 16:53:50 +01:00
j
6fd8d2aef3 fix query for documents rightslevel 2017-11-08 18:00:30 +01:00
j
35c26f095f many parts might block PIPE, use /dev/null 2017-11-08 14:33:35 +01:00
j
7ab94267db common header, custom background color for loading screen 2017-11-06 19:20:37 +02:00
j
8e4ba42fc1 use python3-venv 2017-11-03 17:28:47 +01:00
j
7521467975 split documentsort into it's own manage.py command, does not get applied during install otherwise 2017-10-24 21:28:22 +02:00
j
dbe86cbb99 one more iteritems 2017-10-15 12:00:28 +02:00
j
2eac09691b fix fallback 2017-10-10 17:39:29 +02:00
j
5512252570 clear poster more often 2017-09-21 11:38:12 +02:00
j
85d84c2800 space 2017-09-19 13:16:00 +02:00
j
50c0d67c20 call demo pan.do/ra 2017-08-30 21:20:10 +02:00
j
a687f01fc7 use h264 high profile for .mp4 2017-08-30 14:54:26 +02:00
j
d6ee3019aa fixup b991a3c381 sometimes its a module 2017-08-04 18:55:45 +02:00
j
e1bcb7e348 items move 2017-08-04 18:32:27 +02:00
j
110c57d726 session can be gone 2017-08-04 18:11:51 +02:00
j
b991a3c381 call f.item 2017-08-04 17:34:10 +02:00
j
05534f2de4 fix import 2017-08-04 17:32:36 +02:00
j
ed97ae2423 avoid cache 2017-08-02 17:12:38 +02:00
j
b82f222c73 normalize data to NFC for siteposter, fixes #2428 2017-07-30 11:08:09 +02:00
j
5de40bdfdd user 2017-07-27 21:55:48 +02:00
j
d75f94ffcb typo 2017-07-26 15:36:59 +02:00
j
0043eab61d image might not be ready 2017-07-26 15:33:15 +02:00
j
12cffaf467 only take portrait posters by default from some poster sources, fixes 3034 2017-07-26 13:53:39 +02:00
j
6e7fe215da refactor find 2017-07-26 13:33:08 +02:00
j
41d4e52f84 don't fail on removed layers 2017-07-26 13:33:08 +02:00
j
dacbdf45f8 hide some info about media if user can not see media 2017-07-25 14:58:54 +02:00
j
0df2df3c50 update selected after upload subtitles 2017-07-20 15:35:56 +02:00
j
b7368d74ae fix typo 2017-07-20 15:20:09 +02:00
j
c665b3727c update timeline 2017-07-20 11:13:28 +02:00
j
72cda5596d handle city with , 2017-07-19 19:14:47 +02:00
j
62face2c21 log unknown locations 2017-07-19 18:37:27 +02:00
j
3d100c033c ignore some subtitles 2017-07-19 14:49:18 +02:00
j
fb738c25cf self.id 2017-06-28 12:55:01 +02:00
j
ea256ad4da fix rightslevel titles 2017-06-09 21:30:53 +02:00
j
77810b49d6 add default 2017-06-09 11:29:33 +02:00
j
ac1a567fa4 pass api key 2017-06-09 09:34:49 +02:00
j
ae3afc93c3 space 2017-06-09 09:25:43 +02:00
j
696c4ecaee no torrents for 0xdb 2017-06-06 18:34:47 +02:00
j
c6638cef13 list/map py3 2017-06-06 14:50:51 +02:00
j
4057122d8b mixed a/v timelines 2017-06-01 17:06:22 +02:00
j
fbd94867fc fix documents 2017-06-01 17:00:36 +02:00
j
fb1c42ffaf fix update 2017-06-01 13:23:38 +02:00
j
6d867b1a7d return 404 not 403 for private edits, fixes #3024 2017-05-29 12:14:50 +02:00
j
4fae41c463 extension should always be lowercase 2017-05-28 13:40:56 +02:00
j
90d717769c import Volume 2017-05-28 13:38:42 +02:00
j
772ba2b909 remove unused file 2017-05-26 10:47:40 +02:00
j
d641ea80a8 ignore ctrl-c in py2 wrapper 2017-05-26 10:47:24 +02:00
j
8bd4cd6305 reset video fps/codec 2017-05-21 11:35:54 +02:00
j
f51036ed00 use application/json 2017-05-17 14:22:01 +02:00
j
2717d820a1 update poster 2017-05-03 20:03:07 +02:00
j
0963abac9b stdout is binary 2017-04-28 17:09:12 +02:00
j
a175238d09 fix geo lookup in python3, fixes #3019 2017-04-19 16:34:34 +02:00
j
cb3c18e26b use ui.clipSize 2017-04-17 19:31:53 +02:00
j
4445b33718 add ui.editSidebarSize and update on resize, fixes #2716 2017-04-17 18:18:10 +02:00
j
be34383f8c expose per clip volume in edits 2017-04-15 13:45:31 +02:00
j
28f9a01d90 __str__ 2017-04-15 12:29:12 +02:00
j
18bdc1e6d6 fix typo in config 2017-03-26 10:22:47 +02:00
j
13a166e7f6 cleanup manage.oy 2017-03-16 12:50:28 +01:00
j
87fe056e56 use python3 from venv if manage.py gets started with 2 2017-03-16 12:37:29 +01:00
j
5e37a2d222 add upload to changelog 2017-03-12 22:01:32 +02:00
j
ed7a56a5e9 only cleanup if needed 2017-03-12 14:51:52 +02:00
j
4733dc8994 text can be empty too 2017-03-12 14:50:13 +02:00
j
62ffbca208 pass 0 instead of None 2017-03-11 21:05:52 +02:00
j
d263c4c7f0 position is float 2017-03-11 20:56:37 +02:00
j
7ac8fefcf4 support all metadata keys in add (item) 2017-03-03 18:42:54 +01:00
j
4f14b923fa avoid double entries in accessed table 2017-03-03 18:40:52 +01:00
j
f0a4aba751 use python_2_unicode_compatible 2017-03-03 08:56:35 +01:00
j
72fdc8ed4d fix sort in collection list view 2017-03-02 12:37:53 +01:00
j
19141fe96d fix home editing 2017-02-28 21:32:13 +01:00
j
b3773d4ace fix missing filters 2017-02-28 21:19:26 +01:00
j
50f6c33159 changelog 2017-02-28 20:05:42 +01:00
j
b8f191b3c9 more filter->list(filter 2017-02-25 11:41:48 +01:00
j
67b2fd698b fix save place 2017-02-25 11:18:59 +01:00
j
50bb40291c fix home 2017-02-23 16:32:52 +01:00
j
5947f2295b only delete existing values 2017-02-21 17:46:16 +01:00
j
65c182f410 documentcollection icons 2017-02-20 16:38:23 +01:00
j
efe3ffbed0 fix edit links 2017-02-20 16:13:15 +01:00
j
ebea3b8e11 fix tv cron 2017-02-17 15:50:18 +01:00
j
49c024e87d fix sitemap generation 2017-02-17 15:49:25 +01:00
j
fb74fcc5a7 make sure position is float, fixes #3007 2017-02-17 15:40:53 +01:00
j
2841b65ab4 no -1 items 2017-02-17 15:32:52 +01:00
j
578cfc7799 sitemap 2017-02-16 20:00:50 +01:00
j
2c678ec450 use python for now 2017-02-16 18:31:46 +01:00
j
2f6e7d39c9 fix editing html documents 2017-02-16 18:31:24 +01:00
j
74407183ac fix migration 2017-02-16 18:31:09 +01:00
j
a763291808 fix oembed urls 2017-02-16 14:33:51 +01:00
j
b800110b29 tostring returns bytes... 2017-02-16 14:27:20 +01:00
j
12f3901c9a py3 filter->list(filter 2017-02-16 14:24:51 +01:00
j
2eafeff5f2 manage should use python3 too 2017-02-16 14:12:00 +01:00
j
59c69ba536 default document rightslevels 2017-02-16 14:05:48 +01:00
j
743b419d19 remove unpublished lists/edits/collections from home 2017-02-15 19:07:36 +01:00
j
596d10ce83 home api 2017-02-15 14:08:35 +00:00
j
dc5fd74d9f call home 2017-02-09 03:50:27 +00:00
j
71d72c611d home 2017-02-09 03:55:31 +01:00
j
b055b3e666 only show manage home if its used 2017-02-07 14:32:41 +01:00
j
620c8a7d96 fix document creation 2017-01-27 12:26:55 +00:00
j
85c63d228f manage home dialog 2017-01-26 19:15:00 +01:00
j
390595a261 return current data 2017-01-26 17:06:34 +01:00
j
b23ca9df74 store document references in db 2017-01-26 15:56:28 +00:00
j
df60dca85f return of the document dialog at /document/<id> 2017-01-25 22:01:40 +01:00
j
e4316c9b44 document references 2017-01-25 21:45:54 +01:00
j
0db8ca6c47 document icon, fixes #2987 2017-01-25 17:42:49 +01:00
j
aeab4d03de keep private texts private 2017-01-25 17:39:12 +01:00
j
6e698cf32f document rightslevels 2017-01-25 16:38:38 +01:00
j
ee4d00eaaf home links, inline content 2017-01-25 16:38:25 +01:00
j
ff19404e6d more keys 2017-01-25 14:33:20 +00:00
j
bf8c99513b no access to private documents 2017-01-25 00:13:37 +01:00
j
c5d961a946 home backend 2017-01-24 19:08:32 +01:00
j
c4350e2334 py3 2017-01-24 16:19:34 +01:00
j
7a8c340312 new metadata 2017-01-24 14:44:31 +00:00
j
3b3c22d7ed remove old tv program 2017-01-21 21:12:19 +01:00
j
f71240ac2c look in the right place, fixes #2989 2017-01-20 17:44:31 +01:00
j
bfcf25e240 fold clips 2017-01-20 13:00:04 +01:00
j
e6d21ef0a7 white space 2017-01-18 12:55:47 +01:00
j
b566527483 fix short timelines for multipart videos fixes #2985 2017-01-07 16:52:17 +01:00
j
48587d295c format 2017-01-07 16:50:33 +01:00
j
ce2768e45f lookup name and column in indexdef, fixes ./manage.py sqlfindindex 2016-12-31 23:31:29 +01:00
j
6ad112a3b8 fix pdf upload 2016-11-30 11:36:29 +01:00
j
14bfb1a451 finish tasks without error if item no longer exists 2016-11-30 11:07:38 +01:00
j
cc00510f5d rate limit user cronjob 2016-11-28 14:46:52 +01:00
j
ff4e000954 check against item groups set 2016-11-17 16:09:09 +02:00
j
a8f1c8e20d use canEditMedia instead of hardcoded admin/staff to check file.editable 2016-11-16 11:51:59 +02:00
j
557c078319 fix findMedia by list 2016-11-15 14:46:11 +02:00
j
74c4d718ab fix py3 local_settings 2016-11-15 14:45:54 +02:00
j
b93b2eaf63 make instance times 2038 safe 2016-11-04 17:26:52 +00:00
j
95131a6429 use lower case extensions 2016-10-30 18:23:16 +01:00
j
e1f35b1ec8 new documents section 2016-10-30 18:23:16 +01:00
j
3fcbd59525 try to reload error page after 10 seconds 2016-10-29 16:25:35 +02:00
j
9788c39cd8 do not include protocol and domain in upload url 2016-10-28 18:18:26 +02:00
j
d4a232d594 py3 decode 2016-10-28 18:18:26 +02:00
j
addaa56d85 always clear siteposter cache if poster is removed 2016-10-28 18:15:44 +02:00
j
c19598c4d7 clear siteposter cache too 2016-10-28 17:56:39 +02:00
j
5488eaf223 python3 decode 2016-10-26 22:19:07 +02:00
j
0893715069 add download url for direct uploads 2016-10-10 11:28:21 +02:00
j
2cd89f76ce make cast available in list view, fixes #2971 2016-10-03 13:23:14 +02:00
j
773ab702e7 keys can be None 2016-09-26 10:56:32 +02:00
j
9ef306ab53 support getting related items 2016-09-23 22:58:31 +02:00
j
b7659d6f14 cleanup document cache after upload 2016-09-23 22:58:31 +02:00
j
263b5c9ce2 api py3 fix 2016-09-23 22:57:46 +02:00
j
29600fa5f7 import unquote, fixes #2969 2016-09-20 15:59:49 +02:00
j
23143185ea item can be unavailable 2016-09-20 02:55:06 +02:00
j
55f9d9f58b dont create missing tasks 2016-09-20 01:16:58 +02:00
j
58c762a624 map needed more often, fix cad911a 2016-09-19 20:24:36 +02:00
j
cad911a807 only use -map for audio if needed 2016-09-19 20:05:57 +02:00
j
16d10e1c3f import settings 2016-09-19 19:59:29 +02:00
j
48fae31b62 better debug 2016-09-19 12:04:35 +02:00
j
2c6609b308 typo 2016-09-18 16:31:14 +02:00
j
6c013ebc92 first version of new upload interface 2016-09-15 15:20:57 +00:00
j
8feed54bf9 cancelled->canceled 2016-09-15 15:20:57 +00:00
j
cf0fcdbc5f only paste into small timeline if it exists 2016-09-15 15:20:57 +00:00
j
750633eb7e avoid recomputing timeline for each part 2016-09-15 15:20:57 +00:00
j
285877054e fix document lookup 2016-09-15 13:52:54 +02:00
j
3ac08aa806 make moveMedia async 2016-09-08 00:29:31 +02:00
j
bea661abcf taskId must be in data 2016-09-07 20:23:31 +02:00
j
4d9761a32c make update_external async 2016-09-07 20:16:25 +02:00
j
844d1697bd remove unused annotation.tasks.update_matches 2016-09-06 00:19:36 +02:00
j
4d1711cb01 only update subtitles if needed 2016-09-06 00:13:40 +02:00
j
d53659c0c4 move annotation update_matches logic into models 2016-09-05 23:46:36 +02:00
j
5d43ed0585 Merge changes 2016-09-05 18:17:06 +02:00
j
3cb9281d6d w should never be 0 2016-09-05 18:16:39 +02:00
j
400162e400 full_tile_widths can be 0 for items with many parts 2016-09-05 15:47:36 +02:00
j
bf984fe256 return item info for 0x urls 2016-09-05 13:45:24 +02:00
j
0f0f5f42d8 space 2016-08-31 19:07:44 +02:00
j
1c580a2df7 async update of annotation matches, fixes deadlock in load_subtitles 2016-08-31 18:03:19 +02:00
j
3913373c9b update db if guest was reclassified as robot 2016-08-31 17:38:40 +02:00
j
680efc967c processing trumps queued 2016-08-31 12:23:41 +02:00
j
c409b63f25 __future__ print 2016-08-31 01:33:41 +02:00
j
5ced5cf791 return url 2016-08-31 01:30:13 +02:00
j
89311c14b4 typo 2016-08-31 01:30:13 +02:00
j
cf3161d4bd better findMedia 2016-08-31 01:30:13 +02:00
j
fc6cc93d26 keep import url 2016-08-31 01:30:13 +02:00
j
4d9e8a788a add task for queued downloads 2016-08-31 01:30:13 +02:00
j
1ec69b9c05 more py3 fixes 2016-08-31 00:33:08 +02:00
j
6aabe8d9df py3 fixes 2016-08-31 00:31:38 +02:00
j
9b3547f9bc fix tv 2016-08-28 15:05:52 +02:00
j
78c26daaeb update poster after processing stream 2016-08-28 13:38:46 +02:00
j
f71334ae25 avoid ad 2016-08-28 13:25:26 +02:00
j
da89a9748a include download task 2016-08-28 12:22:31 +02:00
j
029bcfee29 files are queued after download 2016-08-28 12:22:08 +02:00
j
11e36b550e update task after updating item 2016-08-25 18:34:49 +02:00
j
1dd574cb9e task queue has status not state 2016-08-25 18:34:49 +02:00
j
8136a2c2f5 create new item if unregistered file gets uploaded 2016-08-25 18:34:49 +02:00
j
181f27b93a fix audio only uploads 2016-08-25 18:34:49 +02:00
j
1468ddbecb towards supporting python 2 and 3
- use absolute_imports
- make use of six.moves
- use exec instead of execfile
- use list(dict) instead if dict.keys()
2016-08-25 18:34:36 +02:00
j
479432f01a upload_date can be none 2016-08-25 18:32:28 +02:00
j
83c2e25aac support editable as result in find 2016-08-25 18:32:28 +02:00
j
97a46dceb0 fix sort of static edits 2016-08-23 13:24:27 +02:00
j
9685c7e4a5 fix streams for static edits 2016-08-23 13:24:22 +02:00
j
b4074303ba avoid error in error handler and render 500 page 2016-08-23 01:37:38 +02:00
j
d7b53aa322 limit cronjob rate if multiple cronjobs end up in queue 2016-08-18 13:41:24 +02:00
j
e1cacdb67a add task queue api 2016-08-17 14:37:59 +02:00
j
0eb873d2cb fix hue, saturation, lightness calculation for multi part items 2016-08-15 15:14:44 +02:00
j
3d6d1be2e6 include oshash in video url to avoid cache issues 2016-08-13 15:36:00 +02:00
j
843cea063d update timeline if part is changed 2016-08-13 15:34:28 +02:00
j
3b8dd55273 don't fail if 2 paths get flipped in a folder 2016-08-13 15:32:51 +02:00
j
8c928860e1 import url celanup
- only import first url
- pass empty string instead of undefined
- page can contain multiple videos, import as parts
2016-08-10 10:49:58 +02:00
j
b59a9c3122 set canImportItems in config.*.jsonc 2016-08-09 15:58:48 +02:00
j
2157b719ec use more info from sort table, fixes #2953 2016-08-09 15:06:44 +02:00
j
c08ce51db2 add padma importMetadata 2016-08-09 14:25:39 +02:00
j
384c643326 no tags == [] 2016-08-09 14:25:39 +02:00
j
9991767d01 no summary in default importMetadata keys 2016-08-09 13:03:21 +01:00
j
7f8b5448e6 set default config importMetadata 2016-08-09 12:57:43 +01:00
j
0430a0f236 cache getMediaUrlInfo 2016-08-09 12:57:43 +01:00
rlx
90486ec774 update menu and config 2016-08-09 13:35:47 +02:00
rlx
5ef6062613 import media dialog 2016-08-09 13:17:48 +02:00
j
926d3f0f09 add video id 2016-08-09 11:50:24 +01:00
j
337cafcf76 don't add base keys to Item.data 2016-08-08 15:54:52 +02:00
j
9a86ddce01 include upload_date 2016-08-05 17:57:40 +02:00
j
dce51305ce update poster after extracting stream 2016-08-05 15:24:54 +02:00
j
c4a95bc320 update_poster should always update poster 2016-08-05 14:46:27 +02:00
j
2aef460225 do not set -ac if audiochannels is None 2016-08-05 14:46:27 +02:00
j
ff8dd16456 fix profile selection 2016-08-03 11:41:46 +02:00
j
5447aab0a1 more current values 2016-07-30 12:04:35 +02:00
j
dcf3acbd44 store facets in NFKD 2016-07-30 12:01:34 +02:00
j
dffb25e0e8 poster might still point to old id, after rename, delete and make new one. fixes #2944 2016-07-30 10:02:53 +02:00
j
5c90ff79a0 cleanup poster cache 2016-07-30 09:39:31 +02:00
j
893f2e5423 add importMediaDialog 2016-07-30 02:49:43 +02:00
j
383ad8a535 fix findMedia by item id 2016-07-24 17:52:33 +02:00
j
6f4a0ba90a support vtt subtitles 2016-07-01 21:13:13 +02:00
j
f8f951c966 remove outdated fixmes 2016-07-01 17:53:59 +02:00
j
d780045306 flake8 2016-07-01 17:48:43 +02:00
j
3166bf5beb show timeline errors 2016-07-01 17:34:45 +02:00
j
5434ffbf53 make_poster no longer takes arguments 2016-07-01 12:44:07 +02:00
j
88c800a093 use data['positions'] 2016-06-30 16:42:56 +02:00
j
c7157d5001 only decode ids for found positions
merge document.view.get_positions into item.utils.get_positions
add flag to decode id before looking up in ids

Followup to 09ebbc9cc6
2016-06-30 16:18:07 +02:00
j
3d95d7013a Merge remote-tracking branch 'wjt/2935-findDocuments-performance' 2016-06-30 15:43:09 +02:00
j
e9863c238e always use get_operator, avoid case-insensitive match if possible 2016-06-30 11:28:09 +02:00
j
5b545e6e43 layer first 2016-06-30 02:42:24 +02:00
j
b6e4d33a43 add Annotation.findvalue index (used by findClips) 2016-06-30 02:42:24 +02:00
j
b63024df02 show source info if encode fails 2016-06-30 00:23:56 +02:00
j
b5bc17f925 make layer itemKeys with filter=true show up in searching for * 2016-06-30 00:17:23 +02:00
09ebbc9cc6
findDocuments: improve performance of positions queries
References #2935
2016-06-28 16:36:55 +01:00
8d25e3be78
findDocuments: improve entity query performance
When I implemented this in 9a4c24c, there were not many rows in
entity_documentproperties in the database here. Now that there are,
computing the document_document -> entity_documentproperties ->
entity_entity join and then filtering is really, really slow. Postgres
seems to materialize the whole join and then scan it.

If we get a set of matching document IDs for the entity query in a
subquery, and then just filter with IN on that, things are much faster:
scan entity_entity; in a nested loop, get the document_ids via
entity_documentproperties; hash this set; and then scan
document_document.

Searching for a single character, this brings the query from ~1.1s to
~400ms. Searching for a full word, ~800ms to 120ms

This condition is getting really ugly -- I am sorry!

References #2935
2016-06-28 16:33:01 +01:00
j
5aeffcfb6a check first audio track 2016-06-27 16:51:18 +02:00
j
adfcc1cb27 never set display aspect ratio to 0:0 2016-06-27 16:08:30 +02:00
j
8ac78f3bd6 remove unused force flag from make_poster, update_timeline 2016-06-26 23:24:11 +02:00
j
0f9e80e1e6 avoid saving item twice 2016-06-26 23:22:27 +02:00
j
de9b062d63 make sure existing index is using gin 2016-06-26 16:55:58 +02:00
j
ab0dfddf31 set SECURE_PROXY_SSL_HEADER by default 2016-06-26 15:34:19 +02:00
j
0d89ad640b ignore some broken audio codecs 2016-06-26 15:33:52 +02:00
j
92f642cbac pcm sound can have no codec 2016-06-26 14:41:58 +02:00
j
2cec1b9ad5 s/import Image/from PIL import Image/g 2016-06-25 20:39:29 +02:00
j
4785f314cb Add VP9/Opus support, use VP8 by default
- support vp9 and opus
- switch to 2 pass encoding
- use ffmpeg -movflags +faststart instead of qtfaststart
2016-06-23 17:36:41 +02:00
j
aaacc48259 only save if update_external fails 2016-06-20 18:28:05 +02:00
j
d83647c4a5 don't hide oxtimelines errors 2016-06-20 18:27:31 +02:00
j
6dcbcdd19c dont update timeline in update_selected, remove unused async get_item case 2016-06-16 14:48:54 +02:00
j
0486d62ec9 use absolute path 2016-06-16 14:48:09 +02:00
j
f25218466b formating 2016-06-16 14:48:01 +02:00
j
70f34bfde9 typo 2016-06-15 19:13:00 +02:00
j
e3c5ab18c7 only update itemsort if name is changed 2016-06-15 18:31:40 +02:00
j
22f83288c5 avoid looking up item twice 2016-06-15 18:29:09 +02:00
j
7c53dca65b less async item creation 2016-06-15 18:12:59 +02:00
j
b2a9a5f711 space 2016-06-15 17:56:31 +02:00
j
3c1f4a8c95 dont call module 2016-06-15 17:55:57 +02:00
j
b010aca0a9 s/taskId/id/ 2016-06-15 15:45:51 +02:00
j
a0fc6ffadc typo 2016-06-15 14:55:45 +02:00
j
f4cbe6a114 return empty sequences if no data timeline exists 2016-06-15 14:48:02 +02:00
j
af0e0cffe8 person can be removed again, let async itemsort fail without exception 2016-06-15 14:34:46 +02:00
j
fd9d3bdabf flake8 + map->[] 2016-06-15 14:34:46 +02:00
j
05c4cfcbc8 add space and other flake8 cleanups 2016-05-28 11:30:43 +02:00
j
5e149a5cb8 add space and other flake8 cleanups 2016-05-28 11:26:46 +02:00
j
225259e521 add space and other flake8 cleanups 2016-05-28 11:18:51 +02:00
j
f21e8413fb use get_random_string 2016-05-28 11:18:51 +02:00
j
7fdaf6d1ce include Access-Control-Allow-Origin in 404 not found response 2016-05-27 11:51:47 +02:00
05e6118a88
findAnnotations: include duration alongside result count
fixes #2921
2016-05-05 15:54:25 +01:00
j
41cc8e3573 expose encoding status via api 2016-05-05 10:49:34 +02:00
j
be163826ef Merge remote-tracking branch 'wjt/fix-migrations' 2016-05-05 10:48:24 +02:00
39b9b48be2
archive: fix migrations for upload_to function renamings
9c75526 renamed these functions. The function doesn't affect the DB
schema so it should be safe to just modify the migraiton.
2016-05-04 17:01:44 +01:00
e29ea230fb
Add migration for Document.documentproperties ref
This should have been included with a8dcbbb, which changed the
related_name to access DocumentProperties from Document. (There's no
actual change to the database.)
2016-05-04 16:55:11 +01:00
j
0f28a2b7d5 fix queue status 2016-04-30 14:15:13 +02:00
j
9c7552699f fix upload_to callbacks 2016-04-29 13:46:55 +02:00
2812834ce3
findAnnotations: don't lowercase ids (fixes #2916)
Without this fix, a condition like:

     {key: 'id', operator: '==', value: 'A/B'}

gets mapped to:

     public_id__exact=('A/B'.lower())

which is wrong.

I introduced this bug in b3df5b8. I didn't catch it because I was
mostly interested in the 'layer' key -- but layer names are
conventionally lowercase anyway so lowercasing them had no effect.
2016-04-29 11:03:45 +01:00
aa40a40595
Annotation.json: only include entity id & name
Fetching documents for each entity in turn is expensive. (I have tried
using ArrayAgg to fetch them in the same query as the Entity — no
improvement. It's possible that being able to join to entity_entity,
and then use ArrayAgg, would be better.)

Even once you've fetched them all, if the same entity appears many
times in an item, then get(..., keys=['layers']) duplicates the whole
JSON for the entity many times: expensive to serialize, expensive to
send over the wire.

Pandora's own web interface only depends on the 'id' key of 'entity' in
each annotation, and refetches the rest of the entity to show the pop-up
dialog when you press E. So by just not bothering to fetch and send any
other keys, get(..., keys=['layers']) on an item with many entity
annotations is substantially faster.

(I experimented with splitting the full entities off to one side, so,
you'd have:

    {
        "layers": {
            somelayer: [...,
              {..., "entity": {"id": ABC}},
            ], ...
        },
        "entities": {
            ABC: {...},
            ...
        }
    }

This is quicker than the status quo, but obviously not as fast as not
fetching & sending the rest at all!)
2016-04-28 14:15:23 +01:00
aa0fbc9d4a
Entity.json: get document ids from join table
This is a bit quicker because it's just a lookup in a single table, not
a join.
2016-04-28 14:15:12 +01:00
400b6650a2
Annotation.json: document empty-subtitle special case 2016-04-19 13:52:52 +01:00
af0d87b569
Annotation.json: reduce repeated layer lookups
It's actually quite costly to look up keys in CONFIG, particularly
inside a loop: this trims ~5% off get(keys=['layers']) for
annotation-heavy items.
2016-04-19 13:52:47 +01:00
3f5be0bd27
findClips: look up entity names (fixes #2804) 2016-04-19 12:28:58 +01:00
d0129a4416
findClips: avoid O(n²) lookup of clip from annotation
This doesn't make much difference for small ranges, of course.
2016-04-19 11:25:12 +01:00
ba00bcbf7b
findClips: select_related('item') / ('item__sort')
Clip.public_id uses self.item.public_id.

Clip.json() uses self.item.sort, so we should select_related on that
rather than the clip's own sort field. (They are identical objects. Is
Clip.sort ever used directly?)

With this change, findClips() issues one query to fetch clips plus one
query per flavour of annotation; before, it issued two extra queries per
clip.
2016-04-19 11:25:06 +01:00
6dbb7f921a
findClips: only scan layers once 2016-04-19 11:14:25 +01:00
b3df5b8d56 findAnnotations: match some fields case-sensitively
Requiring layer to have the right case is consistent with
addAnnotation(), and means the _layer[_like] index can be used. In my
testing, if itemsQuery specifies a single item, then postgres doesn't
bother with the layer index anyway; but if not, it makes a pretty big
(~3×) difference.

Matching public_id and item__public_id case-sensitively also seems
reasonable (it's consistent with get() and getAnnotation()).

(Is lower() redundant for the case-insensitive comparisons? ie. is
UPPER(x.lower()) == UPPER(x)? I'm not sure, it's cheap, let's leave it.)
2016-04-05 12:19:32 +01:00
8d1b4de337 findAnnotations(): make 'findvalue' the default key
Annotations have no 'name' field, so

     findAnnotations({query: {conditions: [{value: 'foo'}]}})

would previously raise an exception.
2016-04-05 12:19:31 +01:00
284caf03c3 get_by_key: short-circuit
This is about 30% faster, presumably because it avoids allocation and/or
closing over variables is slow(?). It's not hugely significant (I
misread a line_profile report) but why not.
2016-04-05 12:19:31 +01:00
j
7ac68697d4 update pdf.js 2016-04-04 15:50:07 +02:00
j
652df88342 return 404 2016-04-04 15:50:07 +02:00
j
1bff4aa0e9 avoid storing invalid poster frames, only show videos with video 2016-04-01 16:40:20 +02:00
j
b8beb51480 fix multipart audio only timelines 2016-03-31 14:54:38 +02:00
j
94b940436f fix timelines for items with many parts
- use durations from streams not from timelines
 - don't accumulate timeline drift
2016-03-19 18:58:48 +01:00
j
f0b8b2b81e check that range is [int, int] 2016-03-17 16:06:08 +01:00
eeaeda3970 Support WebVTT subtitle export 2016-03-11 14:16:23 +01:00
j
697e501a4f only update item timeline once all parts are done 2016-03-11 10:33:48 +01:00
j
f6cebcaec9 fix user/group api 2016-03-08 20:14:05 +05:30
j
29204b6fb5 move gunicorn configuration from init script to config file 2016-03-07 14:25:24 +05:30
j
4f28c2c548 fix annotation import, values are decoded in d1.9 2016-03-05 15:36:47 +05:30
a8dcbbbe89 Include DocumentProperties.data in Document.json() 2016-03-05 15:07:47 +05:30
a55cbcfb9f DocumentProperties: add data field 2016-03-05 15:07:47 +05:30
0c98cd080e Entity.alternativeNames: default to () not [] (fixes #2896)
Otherwise this:

    self.name_find = '||' + '||'.join((self.name,) + self.alternativeNames) + '||'
                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

fails because () + [] is an error. I guess this must have been
introduced by the DictField/TupleField rewrite.

Without this fix, it is impossible to create a new entity.

Basically the same logic is used for Event and Place too so I've made
the same change to those, and, in passing, fix another copy of the bug
fixed for Entity.name_find in fe7f961.
2016-03-04 17:11:36 +00:00
9a4c24cdb4 Support searching documents by entities 2016-03-04 12:41:41 +00:00
738a9282b4 Document: fix negating id queries 2016-03-04 12:41:41 +00:00
8c23bdff6d Implement DocumentProperties.__unicode__ 2016-03-04 12:41:41 +00:00
j
4613005b83 use geoip2 api to fix ipv6 lookups 2016-03-04 12:50:44 +05:30
c6f9f87c8e Fix autocompleteSort with multiple keys (fixes #2893)
QuerySet.order_by() takes each key as a separate argument, not as a
single comma-separated string.
2016-03-03 18:15:37 +05:30
2a07e2a1ab Remove redundant overrides of Model.delete
Both of these models have pre_delete handlers which do the same things,
so I think these are unnecessary.
2016-03-03 18:10:29 +05:30
d69a8efd97 Don't save other file-owning models on delete, either 2016-03-03 18:10:29 +05:30
6e0049a20c Don't save Document in pre_delete handler (fixes #2889)
FileField.delete() will, by default, save() the model instance it is
attached to. This is pointless if we're in the process of deleting the
Document -- and since Document.save() calls Document.update_matches(),
this scans all annotations every time a document is deleted.
2016-03-03 18:10:29 +05:30
j
fd2c69a5b7 define BASE_DIR 2016-03-03 16:56:06 +05:30
j
11924fcb0d Rename api.taskStatus to api.getTaskStatus
still register api.taskStatus since its used
in pandora_client and other scripts.

fixes #2600 and fixes #2859
2016-02-20 18:35:26 +05:30
j
0d5c968026 space 2016-02-20 10:41:23 +00:00
j
051a3578b4 inline django 1.9 version of ox.django 2016-02-20 10:41:23 +00:00
j
7620795862 add update_itemsfolder command 2016-02-20 10:41:22 +00:00
j
0367816257 only crop with 4 crop points 2016-02-20 10:41:22 +00:00
j
1ef547f885 update celery integration 2016-02-20 10:41:22 +00:00