tobias/radicale
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
1,323 Commits 2 Branches 0 Tags 3 MiB
0601b81aa7
Commit Graph

980 Commits

Author SHA1 Message Date
Unrud
d5b8ddd71c Check that name is valid in name_from_path 
Before it was possible craft XML requests, so that the storage backend got requests with invalid hrefs.
 2016-09-04 22:28:01 +02:00
Unrud
139076faee Sanitize URLs from XML requests 2016-09-04 22:27:59 +02:00
Unrud
13d652b094 Remove unnecessary module prefix 2016-09-04 22:26:46 +02:00
Unrud
664fa71278 Don't double unquote request URL 
"%2525" was transformed to "%" instead of "%25".
 2016-09-04 22:26:46 +02:00
Unrud
dbaf58dbfe Remove base_prefix and use SCRIPT_NAME instead 
This conforms with the WSGI reference (PEP 333)
 2016-09-04 22:26:40 +02:00
Unrud
03fbb1e68e Don't strip {} in get method 
If someone uploads a file that starts or ends with the chars {}, all REPORT requests on that collection will fail and it's impossible to delete the file.
 2016-09-04 13:35:44 +02:00
Unrud
6df54bf88a Log name of faulty component 
If vobject can't parse a component it raises an exception, but the filename of that component is missing in the logs.
 2016-09-04 13:23:01 +02:00
Unrud
5ccfe16372 Remove Collection.has 
It's the same as BaseCollection.has
 2016-09-04 13:21:57 +02:00
Unrud
de09f6689a Only relevant files for last_modified calculation 
Leftovers from failed transactions etc. should not change that property.
 2016-09-04 13:16:42 +02:00
Unrud
cd9f789294 Name variables for files f 
fd sounds more like file descriptions.
prop doesn't sound like a file at all.
 2016-09-04 13:14:51 +02:00
Unrud
f5650df5f7 Remove checks for existence of collection 
They are unnecessary since the discover methods stopped returning collections that actually don't exist.
 2016-09-04 13:13:35 +02:00
Unrud
e7d8b4816c Duplicate code: Use list and get methods 2016-09-04 13:12:55 +02:00
Unrud
5dbf9df876 Add missing checks for safe fileystem components 
Currently it's not possible to exploit these.
 2016-09-04 13:09:10 +02:00
Unrud
dc501d5dc5 Refactor/Duplicate code: Extract _fsync method 2016-09-04 13:08:12 +02:00
Unrud
77e9ca1252 Remove EtagMismatchError 
Etags are not checked in storage anymore and this is unused.
 2016-09-04 13:06:09 +02:00
Unrud
a12ef69129 Secure is_safe_filesystem_path_component 
On Windows 1/2 would be a safe filesystem path component, but it's not safe to pass it to path_to_filesystem.
Currently only the get method can be called with a href like that and it checked for that.
This just moves the check into the is_safe_filesystem_path_component function.
 2016-09-04 12:55:28 +02:00
Unrud
a4a6a62643 Duplicate code: Use is_safe_path_component 2016-09-04 12:53:07 +02:00
Unrud
8d5f2ded42 Describe encoding of Etag 2016-09-04 12:52:29 +02:00
Unrud
d371179487 thread and level in simple log formatter 
The log messages are a mess without any allocation to threads.
 2016-09-04 11:47:27 +02:00
Unrud
fe5daf801a Set password to empty string instead of None 
Prevent exception in auth module.
 2016-09-03 10:01:52 +02:00
Guillaume Ayoub
b85fc5bed6 Merge pull request #507 from Unrud/auth 
Repair and test authentication
 2016-09-02 18:36:31 +02:00
Unrud
9e27d4e2a8 Emulate fullmatch with match 
re.fullmatch was introduced in Python 3.4
 2016-09-02 15:06:32 +02:00
Unrud
b25a601e28 Test custom header 2016-09-02 14:52:45 +02:00
Unrud
11df2f1184 Test authentication 
Test for 2a9f37defb
 2016-09-02 14:42:22 +02:00
Unrud
2a9f37defb Repair authentication 2016-09-02 14:41:31 +02:00
Guillaume Ayoub
88d558f9fa Remove useless nested tuples 2016-09-02 11:05:35 +02:00
Guillaume Ayoub
68e1e9dfb2 Don't use mutables in constants or parameters 2016-09-02 11:04:29 +02:00
Unrud
20b1480399 Make copy of headers before mutating 2016-09-02 04:23:47 +02:00
Unrud
f7e995f9f6 Move encoding of answer into response function 
Fix #505
 2016-09-02 04:10:11 +02:00
Unrud
3df367aad2 Test REPORT on item 
Test for 90e4655a44
 2016-09-01 06:17:18 +02:00
Unrud
9d2743d797 Test authorization 2016-09-01 06:15:31 +02:00
Guillaume Ayoub
e25373fa85 Merge pull request #501 from Unrud/httperrors 
HTTP error messages
 2016-08-31 14:49:19 +02:00
Guillaume Ayoub
8d92b371e4 Merge pull request #502 from Unrud/patch-40 
Plain text for GET / instead of malformed HTML
 2016-08-31 14:42:59 +02:00
Guillaume Ayoub
db681da08e Merge pull request #500 from Unrud/loginuser 
Map logins to internal users in Auth module
 2016-08-31 14:40:08 +02:00
Unrud
f875bcd892 Plain text for GET / instead of malformed HTML 
It's basically the same in browsers and looks nicer in tools that don't support HTML rendering (like curl).
 2016-08-31 02:01:18 +02:00
Unrud
ff2b8f6e5c Use NOT_FOUND instead of GONE 
Thunderbird doesn't recognize the status code correctly and shows an synchronization error.
 2016-08-31 01:54:31 +02:00
Unrud
1ea9b1dca9 Return error for GET requests on directories 
Radicale doesn't support directory listings.
 2016-08-31 00:45:14 +02:00
Unrud
4ac2e68f5c Add HTTP error messages 
Browsers just show a blank page if an error occurs. You have to open the developer tools to see the HTTP status code. E.g. a user wants to download a calendar in the browser and the URL is wrong.

Some tools like curl don't show any indication of an error.
 2016-08-31 00:41:08 +02:00
Unrud
689e5c9dd5 Map logins to internal users in Auth module 
This makes it possible to implement #349 as a Auth module. Another use case would be to encode usernames that contain characters unsupported by the file system.
 2016-08-30 23:13:33 +02:00
Unrud
e782808a14 Remove empty line in log 
Separating requests by an empty line doesn't work any longer with parallel (overlapping) requests.
 2016-08-30 14:04:16 +02:00
Markus Unterwaditzer
90e4655a44 Don't return <item>/ for events 2016-08-29 17:29:46 +02:00
Guillaume Ayoub
947c57312c Fix a variable name 
Fix #494
 2016-08-29 12:09:17 +02:00
Guillaume Ayoub
34ad1b9073 Remove useless import 2016-08-29 12:07:58 +02:00
Guillaume Ayoub
fd1742fcea Fix typo 2016-08-29 12:07:30 +02:00
Guillaume Ayoub
9e78454da2 Merge branch 'remupdate' of https://github.com/Unrud/Radicale into Unrud-remupdate 2016-08-26 22:50:26 +02:00
Guillaume Ayoub
a3dcfaacd6 Merge pull request #491 from Unrud/uploadall 
Optimize upload of whole collections
 2016-08-26 22:34:46 +02:00
Guillaume Ayoub
ac61b18237 Merge pull request #490 from Unrud/hook 
Move hook into storage.Collection
 2016-08-26 15:58:42 +02:00
Guillaume Ayoub
0e646cdae4 Merge branch 'Unrud-logging' 2016-08-25 11:52:12 +02:00
Unrud
c5342d36d5 Remove BaseCollection.update 
I don't think that this can be used for optimizations.

It's useless in the filesystem backend, SQL has REPLACE and I doubt that there is much use in any other storage mechanism.
 2016-08-25 06:37:12 +02:00
Unrud
ea63f461a8 Rename upload_all to upload_all_nonatomic 2016-08-25 06:03:13 +02:00
Unrud
30d287ce00 Write files nonatomic in upload_all 
It's only used in temporary collections.
 2016-08-25 06:03:13 +02:00
Unrud
e31ea57883 Use upload_all for addressbook 2016-08-25 06:03:13 +02:00
Unrud
bc0f8b0a47 Remove duplicate code 2016-08-25 06:03:13 +02:00
Unrud
c57307c585 Rename collections to vobject_items 
Like the parameter name of upload_all
 2016-08-25 05:52:26 +02:00
Unrud
6045ad97fe Move upload_all from BaseCollection to Collection 
This is not used anywhere outside of Collection and probably never will be as WebDAV doesn't support bulk uploads.
 2016-08-25 05:47:31 +02:00
Unrud
10786cbad8 Move hook into storage.Collection 
The hook is only valid for filesystem storage, it's meaningless for other backends like databases.
 2016-08-25 05:40:46 +02:00
Unrud
8db580abce Try to decode URLs with utf-8 (Fixes #486) 2016-08-25 05:30:46 +02:00
Unrud
e40e46e164 Don't disable existing loggers 
The logger is retrieved before configure_from_file is called and gets disabled, the same happens when the logging configuration is reloaded.
 2016-08-25 05:24:29 +02:00
Unrud
de8c2f0909 Fix SIGHUP handler 
The function handler_generator seems useless and the return statement is missing.
 2016-08-25 05:24:29 +02:00
Unrud
3b71ab960e Log exceptions (Fixes #447) 
Exceptions were just written to stderr but not into logs.
 2016-08-25 05:24:24 +02:00
Unrud
c091399f5e Write log to stderr 
Be consistent with python's default behavior and play nice with CGI.
 2016-08-25 05:19:38 +02:00
Unrud
14515cfe27 Fix logger configuration 
Apply patch from #485
 2016-08-25 04:29:02 +02:00
Guillaume Ayoub
79bfa9c1d3 Merge pull request #482 from Unrud/smallimprovements 
Some small cosmetic improvements for xmlutils
 2016-08-24 18:11:06 +02:00
Unrud
f9cabfb20b Test empty filter 
Test for d1dbd1df54
 2016-08-24 17:52:39 +02:00
Unrud
d1dbd1df54 Fix empty filter 
DAVdroid sends <CARD:filter />, which causes an exception.
 2016-08-24 17:45:10 +02:00
Unrud
469efbb032 Cosmetics 2016-08-13 04:52:51 +02:00
Unrud
453a8ba636 Fix "fix this" in xmlutils 2016-08-13 04:51:42 +02:00
Unrud
ecd0a16214 Remove unused variable path 2016-08-13 04:48:59 +02:00
Unrud
1c6e626979 Simplify name_from_path and add error check 2016-08-13 04:47:35 +02:00
Guillaume Ayoub
3b29a56c81 Merge pull request #480 from Unrud/currentuserprincipal 
Ask for authentication if current-user-principal is requested
 2016-08-13 04:25:55 +02:00
Unrud
c29fd1ca61 Check that item is a collection 2016-08-13 00:19:14 +02:00
Unrud
cf4a6ef749 Revert 1ffc171f35 
Maybe I misunderstand the RFC, but this properties are related to a principal collection. DAVdroid without preemptive authentication doesn't like this and tries to create calendars and addressbooks in /.
 2016-08-13 00:17:50 +02:00
Unrud
4438d2ba9a Require user for current-user-principal 
DAVdroid dropped support for preemptive authentication in version 1.2.3.
Returning the DAV:unauthenticated pseudo-principal as specified in RFC 5397 doesn't seem to work for DAVdroid.
 2016-08-12 23:41:34 +02:00
Unrud
6515062bcd Return HTTP status in xmlutils.propfind 2016-08-12 23:34:08 +02:00
Florian Mounier
c95437367f This is very important 2016-08-12 15:05:05 +02:00
Florian Mounier
416f9ddef1 Add an upload_all method for subsequent optimizations. 2016-08-12 14:58:32 +02:00
Guillaume Ayoub
8d863e52b2 Re-enable properties storage on al collections 
Fix #475.
 2016-08-11 12:07:19 +02:00
Guillaume Ayoub
d3d29802ad Merge pull request #474 from Unrud/patch-31 
PROPFIND rights checking
 2016-08-11 11:44:43 +02:00
Guillaume Ayoub
e66a35e996 Merge pull request #476 from Unrud/closelockfile 
Add option to close lock file
 2016-08-11 11:43:40 +02:00
Guillaume Ayoub
f395e256ff Merge pull request #478 from Unrud/patch-33 
Correctly write line endings to files
 2016-08-11 11:43:18 +02:00
Unrud
9e84b459ae Correctly write line endings to files 
vobject uses \r\n as line endings. Writing this to a file is not a problem on Linux and newer versions of MacOS. On Windows \r\r\n gets written to disk and on older versions of MacOS \r\r gets written to disk, because python replaces \n by the system depended line ending.
 2016-08-11 05:43:18 +02:00
Unrud
7a01f905de Enable close_lock_file for tests 2016-08-11 05:20:31 +02:00
Unrud
3f5dd70580 Add option to close lock file 
Close the lock file, when no more clients are waiting.
This option is not very useful in general, but on Windows files that are opened cannot be deleted. This causes tests to fail, because the deletion of the temporary filesystem folder fails.
 2016-08-11 05:19:23 +02:00
Unrud
0060130c3b Remove etags from raw data 
Remove all etags that are directly calculated from data that's read from files.
1. They are not used anywhere (luckily).
2. Etags that are send to clients are calculated from the output of vobject's serialize method. If files are edited externally and vobject normalizes them (like wrapping long lines or replacing all line endings by \r\n), the etags that are sent to the client and the etags that are calculated from raw data will never match. If a new version of vobject is released and the formatting changes slightly, the checks will also always fail.
 2016-08-11 05:05:10 +02:00
Guillaume Ayoub
35d12ee97e Merge pull request #472 from Unrud/patch-30 
Use os.replace instead of os.rename
 2016-08-11 04:23:56 +02:00
Guillaume Ayoub
69386410e5 Merge pull request #473 from Unrud/tests 
Tests
 2016-08-11 04:23:23 +02:00
Unrud
4eb04e3526 PROPFIND rights checking 
Return 404 and 403 only when it's appropriate. Don't ask users for passwords if an item just doesn't exist (e.g. mistyped URL).
 2016-08-11 02:19:48 +02:00
Unrud
30b3273efa Test that the requests fails if the hook fails 2016-08-11 00:37:19 +02:00
Unrud
2b45cffa0e Test that hook gets executed when the principal collection is created 
Test for ee5b8facda
 2016-08-11 00:37:19 +02:00
Unrud
080ed31d27 Test that storage is locked when hook runs 
Test for 65af0592d3
 2016-08-11 00:37:19 +02:00
Unrud
952609deee Test that hook gets executed on write accesses 2016-08-11 00:37:19 +02:00
Unrud
893051645e Test fsync 
Enable syncing for at least one test.
Test for 5c2075cb6c.
 2016-08-11 00:37:19 +02:00
Unrud
6d80b70b8c Test that the root collection always exists 
Test for 6c3e59fd11
 2016-08-11 00:37:19 +02:00
Unrud
97edacd71a Test implicit creation of principal collection 
Test for 81b04890f1
 2016-08-11 00:37:19 +02:00
Unrud
62892e3423 Test PROPPATCH 2016-08-11 00:37:13 +02:00
Unrud
51bf95f00d Don't run all tests twice 
Only verify that custom backend loading works with a simple test.
 2016-08-11 00:32:24 +02:00
Unrud
c8e8993ec2 Refactor: Move common code into BaseFileSystemTest 2016-08-11 00:32:24 +02:00
Unrud
e2b87d145f Cosmetics: Don't use % for logging 2016-08-10 23:43:32 +02:00
Unrud
9192a7751b Remove incorrect argument 
In rare cases this can cause a crash.
 2016-08-10 23:41:19 +02:00
Unrud
b8126f8d24 Use os.replace instead of os.rename 
On Windows os.replace sets the MOVEFILE_REPLACE_EXISTING flag for MoveFileEx.
On POSIX it's the same as os.rename.
 2016-08-10 19:26:07 +02:00
Guillaume Ayoub
5c2075cb6c Fix _atomic_write 2016-08-08 14:55:01 +02:00
Guillaume Ayoub
5e5b8b844f Cosmetics 2016-08-08 13:39:01 +02:00
Guillaume Ayoub
1e5c9f63a0 Merge pull request #468 from Unrud/disablefsync 
Add option to disable syncing to disk
 2016-08-08 13:00:12 +02:00
Guillaume Ayoub
c4cf918bf2 Merge pull request #470 from Unrud/readcontent 
Read content after access checks
 2016-08-08 12:57:59 +02:00
Unrud
a9b89be5c7 Read content after access checks 
Unauthorized users can't fill up RAM with crap anymore.
 2016-08-08 07:00:24 +02:00
Unrud
f294b1cf17 Add access check to PROPFIND 2016-08-08 06:59:15 +02:00
Unrud
eb15de0c5b Test PUT with whole collection 2016-08-08 06:09:24 +02:00
Unrud
68286faa63 Atomic replacement of whole collection by PUT 2016-08-08 06:08:52 +02:00
Unrud
0675328a02 Replace collection in Collection.create_collection 2016-08-08 06:08:01 +02:00
Unrud
6d85a731e5 Disable syncing to disk for tests 
This reduces test time by almost 70%.
 2016-08-08 05:30:16 +02:00
Unrud
f5f52582a1 Add option to disable syncing to disk 
Disabling syncing increases the risk of data loss when the system crashes or power fails. On the positive it can increase the performance to a great extent.
 2016-08-08 05:20:25 +02:00
Unrud
c336e0581e Remove atomicwrites 
Unfortunately the library doesn't support disabling of disk syncing, fortunately we only need a small subset of it's functionality which is easy to implement.
 2016-08-08 05:02:36 +02:00
Unrud
3c736cade8 Refactor: Move sync_directory into Collection class 
This is not used anywhere else.
 2016-08-08 04:07:01 +02:00
Guillaume Ayoub
9f2cbb81a3 Merge pull request #466 from Unrud/fixpath 
Set correct path for child collections
 2016-08-07 18:43:41 +02:00
Unrud
ce0a2fd01d Test PROPFIND 
Regression test for cfa8c7d8b4
 2016-08-07 18:08:10 +02:00
Unrud
cfa8c7d8b4 Set correct path for child collections 2016-08-07 17:50:31 +02:00
Unrud
4f37e90e20 Some clients expect collections to end with / 
Compatibility with InfCloud/CalDavZAP/CardDavMATE
 2016-08-07 17:50:04 +02:00
Unrud
ee5b8facda Always use wrapper to lock collection 
Also run the hook after creation of the principal collection.
 2016-08-07 17:14:47 +02:00
Guillaume Ayoub
4236077b04 Document _makedirs_synced and remove exist_ok 2016-08-06 14:08:21 +02:00
Guillaume Ayoub
4549d1b2db Merge pull request #462 from Unrud/durabledirs 
Durable creation of directories and make sure that the root colleciton exists.
 2016-08-06 14:02:41 +02:00
Guillaume Ayoub
a7923008ce Remove unused import 2016-08-06 13:29:07 +02:00
Guillaume Ayoub
0cda9f611d Merge pull request #463 from Unrud/atomicmove 
Atomic MOVE
 2016-08-06 13:27:38 +02:00
Guillaume Ayoub
071a829af8 Merge pull request #453 from Unrud/delete 
Delete atomic and durable
 2016-08-06 12:52:38 +02:00
Unrud
5f66d009d6 Atomic MOVE in multifilesystem 2016-08-06 05:09:06 +02:00
Unrud
23582c8208 Expose low level MOVE operation in storage.BaseCollection 2016-08-06 05:09:00 +02:00
Unrud
17ff22cae4 Support replacing in MOVE method 2016-08-06 04:45:44 +02:00
Unrud
89ac2fb397 Remove faulty check 
This should have been (to_path.strip("/")+"/").startswith(path.strip("/")+"/").
But it's not required as we don't support moving collections.
 2016-08-06 04:44:18 +02:00
Unrud
07dc71fd73 Update comment 2016-08-06 04:07:55 +02:00
Unrud
6c3e59fd11 Make sure that the root collection exists. 
Since the collections are not directly in **filesystem_path** anymore,
the folder is not created by ``Collection.acquire_lock``.
 2016-08-06 04:07:55 +02:00
Unrud
05a4285f54 Durable creation of intermediate directories 2016-08-06 04:07:50 +02:00
Guillaume Ayoub
9aefc500ec Merge pull request #461 from Unrud/patch-27 
Remove old code
 2016-08-05 17:48:03 +02:00
Unrud
bfb7aec7be Test deletion of root collection 2016-08-05 17:40:47 +02:00
Unrud
f4eb143ba8 use tempfile to delete collection 2016-08-05 17:40:47 +02:00
Unrud
5a9d956b49 delete atomic and durable 
See #440
 2016-08-05 17:40:47 +02:00
Unrud
9dd241a54b Add test for deletion of collection 2016-08-05 17:40:47 +02:00
Unrud
6d77e24028 Remove old code 
This was missed while merging #457 and #459.
 2016-08-05 17:05:32 +02:00
Unrud
65af0592d3 Run hook while storage is still locked 2016-08-05 16:58:03 +02:00
Guillaume Ayoub
50438bc2c0 Remove fixed TODOs 2016-08-05 02:57:17 +02:00
Guillaume Ayoub
8682504c6e Strip empty lines from content 2016-08-05 02:25:15 +02:00
Guillaume Ayoub
26e1c9ba89 Fix collection creation with PUT request 2016-08-05 02:24:52 +02:00
Guillaume Ayoub
8ac3ce1a89 Clean many, many things 2016-08-05 02:14:49 +02:00
Guillaume Ayoub
92a0027ae1 Merge branch 'rights' of https://github.com/Unrud/Radicale into Unrud-rights 2016-08-04 23:35:01 +02:00
Guillaume Ayoub
2eaedf448f Merge pull request #457 from Unrud/atomiccreate 
Atomic creation of collections and atomic PROPPATCH
 2016-08-04 22:55:15 +02:00
Unrud
8ce6d1af30 Use flock locks for storage locking 
These locks are compatible with the command line utility flock,
which comes preinstalled with most Linux distributions.
 2016-08-04 06:15:05 +02:00
Unrud
066b5994d1 Improve rights checking and request handlers 
* Access rights are checked before the storage is locked and
    collections are loaded.
  * DELETE sends 410 instead of doing nothing or crashing if the target
    doesn't exist.
  * GET always returns 404 if the target doesn't exist.
  * GET doesn't crash if a collection without tag property is requested.
  * MKCOL and MKCALENDAR send 409 if the target already exists.
  * MOVE checks if the target collection of an item actually exists and
    sends 409 otherwise.
  * PUT doesn't crash if a whole collection that doesn't exist yet is
    uploaded and ``content-type`` is ``text/vcard`` or
    ``text/calendar``.
  * PUT distinguishes between simple items and whole collections by the
    following criteria: Target is a collection; Parent exists; Parent
    has the tag property set; Parent contains other items. Before only
    the first two criteria where used, which was very unrelieable. #384
  * PROPPATCH is only allowed on collections and 409 is send otherwise.
  * ``Rights.authorized`` takes a path instead of a collection.
  * ``Collection.discover`` only returns items in ``path``, that
    actually exist. #442
 2016-08-04 06:08:08 +02:00
Unrud
ae89082c24 Atomic creation of collections 2016-08-03 15:50:55 +02:00
Unrud
e34d1c46cd Move collections into collection-root folder 
This is required for atomic creation and deletion of the "/" collection.
 2016-08-03 15:50:38 +02:00
Unrud
bd7641699e Atomic PROPPATCH 2016-08-03 14:45:52 +02:00
Unrud
de510148a0 *args and **kwargs for test collection 2016-08-03 14:35:50 +02:00
Unrud
0fc7f787a8 Remove test_folder configuration 
It's not used.
 2016-08-03 14:34:36 +02:00