Commit Graph

1086 Commits

Author SHA1 Message Date
Unrud
30a4105065 Add TODO comments for time ranges 2017-06-07 21:54:49 +02:00
Unrud
bab35e9692 Detect errors in recurrence rules early 2017-06-07 21:54:38 +02:00
Unrud
0e35ab1baf Fix detection of infinite recurrence
* The RRULE field might not exist.
* vobject also accepts lower case COUNT and UNTIL.
2017-06-07 15:31:05 +02:00
Unrud
3753364fc8 XML error message for invalid sync-token 2017-06-07 14:17:57 +02:00
Unrud
13381fb829 Change logging level of invalid sync token to warning
Some clients stop to work, when told their sync-token are invalid.
2017-06-07 14:15:30 +02:00
Unrud
1b54b23bb3 Add info about disabling locking to error message 2017-06-07 14:14:09 +02:00
Unrud
217859bf5e Fix empty REPORT requests 2017-06-07 14:13:12 +02:00
Unrud
2860c664d0 Check that vobject_item have a UID 2017-06-06 20:01:09 +02:00
Unrud
fe97741f08 Better reporting of errors in PUT requests 2017-06-06 20:01:07 +02:00
Unrud
e47747d4d4 return None instead of False if UID field is missing 2017-06-06 20:01:06 +02:00
Unrud
bea855cb80 Improve file names used when uploading whole collection
* Use 64 bit random sequence (extremely low chance of collisions)
* Improve error message in case of collisions
* Add file extension to names
2017-06-06 20:01:04 +02:00
Unrud
a6a3756e05 Check error code of PUT request
Detect errors early
2017-06-06 20:01:02 +02:00
Unrud
5660f70753 Rename deleted item in test
The backend may derive the item name from the UID (e.g. "event1.ics" from the UID "event1").
2017-06-06 20:00:59 +02:00
Unrud
ee214d6697 Remove user from URL
It's not supported by many clients.
2017-06-04 15:43:55 +02:00
Unrud
37c9a1da18 Store user and password in session storage
Allow reloading of the page.
2017-06-04 15:43:27 +02:00
Unrud
4afbefbc9f Disable update timer 2017-06-04 15:43:20 +02:00
Unrud
fca70f482f Web interface: Wrapping of URLs, titles, ... 2017-06-04 15:42:33 +02:00
Unrud
396b9ffecb Log error message from parser for items in storage 2017-06-02 16:36:02 +02:00
Unrud
a1ebc52b39 Don't delete cache entry that gets overwritten
It's unnecessary.
2017-06-02 14:19:53 +02:00
Unrud
07c792f14e Use %r n error messages for token names 2017-06-02 14:18:39 +02:00
Unrud
9ceae0a751 Make Radicale fast (#569)
* Change get_multi to also return missing items

get_multi is not used anywhere and this makes it easier to use.

* Use get_multi for report requests

* Add get_all to BaseCollection

This can be used for optimization on multifilesystem.

* Use iterator for files

* Remove unnecessary checks

This never happens and would be an error.

* Don't raise exception when calling get with colliding name

This behavior is wrong, it should be handled as if the file doesn't exist.

* Use get_all and get_multi to skip unnecessary checks

Collision checks are slow on big collections.

* Use exception instead of existence checks

It's a bit faster.

* Use os.scandir instead of os.listdir

It's faster and doesn't load all files at once.

* Cache metadata when storage is read-only

Metadata is queried a lot during a request. It's quiet slow to load and parse the file every time.

* Cache the etag when the storage is read-only

The etag is calculated twice for GET requests on collections.

* Add helper method for cleaning caches

* Use item etags to calculate collection etag

It's very slow and unnecessary to parse all files with VObject and serialize them again.

* Cache serialized collections in file system

Serialization is very slow for big collections. This caches the result in a file.

* Add helper function for prefilters

The simplify_prefilters functions converts XML filters to a simple tag and time range, which can be easily matched against the tag and time range that are extracted from vobject_items by the function find_tag_and_time_range.

* Add ability to cache etag and serialization of item

Parsing items with vobject is very slow and not required for many requests.
Caching can be used to speed it up.

* Cache metadata and serialization from items in file system

Store the serialized text and the tag and time range from vobject_items in the cache.
The metadata is used for prefilters.

* Remove the cache for the serialization of collections

* Serialize calendars without vobject

Merge the calendar components manually. This is much faster and requires less memory. Caching of the result is not required anymore.

* Allow pre_filtered_list to indicate that filters match

The storage backend can indicate that it evaluated the filters completely.

* Skip filtering with vobject if prefiltering is sufficient

``simplify_prefilters`` indicates if the simplified condition is identical to ``filters``.
This is used in the multifilesystem backend to detect if prefiltering is sufficient.

* Make constants global

* Use generator expressions

* Only extract elements from inside of VCALENDAR

This is unnecessary at the moment, the text representation should never contain anything but VCALENDAR.

* Improve comments

* restore backward compatiblity

* Small improvements for fastbackend
2017-06-02 14:14:55 +02:00
Unrud
78a62aee86 Merge pull request #565 from Unrud/synctoken
Support for sync-token and sync-collection
2017-06-02 13:01:54 +02:00
Unrud
7687afc4c6 Merge pull request #622 from Unrud/clientcertificate
Add option for CA certificate for validating clients
2017-06-02 12:59:35 +02:00
Unrud
f633b48a7a Real sync-collection support for the multifilesystem backend 2017-06-02 12:53:37 +02:00
Unrud
0ffa97eca0 Allow opening binary files with _atomic_write 2017-06-02 12:44:36 +02:00
Unrud
1dee561692 Add helper method for cleaning caches 2017-06-02 12:44:34 +02:00
Unrud
3009ce5414 Test sync-token and sync-collection 2017-06-02 12:44:31 +02:00
Unrud
f2b415c4a6 Initial sync-token and sync-collection support
Use the etag of the collection as the sync token and tell the client that the token is invalid when the collection changed.
2017-06-02 12:44:23 +02:00
Unrud
428abf10de don't use REMOTE_USER in tests 2017-06-02 12:43:44 +02:00
Unrud
6edaf27a38 rename backend from "None" to "none" 2017-06-02 12:43:23 +02:00
Unrud
5d27265d5c fail when logging config file is not found 2017-06-02 12:43:03 +02:00
Unrud
1812aeb238 include expected type of config option in error 2017-06-02 12:42:40 +02:00
Unrud
881757815f Add simple range checking to config options 2017-06-02 12:42:19 +02:00
Unrud
95a8c7b903 use mapping api to set config options
Provides protection against typos in names
2017-06-02 12:41:47 +02:00
Unrud
3af5809d71 Add option for CA certificate for validating clients
This can be used to secure TCP traffic between Radicale and a reverse proxy
2017-06-02 12:41:03 +02:00
Unrud
3bfac019a8 Close lock file when running tests
The name of configuration option was wrong.
2017-06-01 16:10:50 +02:00
Unrud
c400414b27 Don't turn off all locking with filesystem_locking 2017-06-01 16:10:34 +02:00
Unrud
f715f0ba8a Remove font from web interface 2017-06-01 16:10:17 +02:00
Unrud
ac230efa57 Merge pull request #619 from Unrud/log_unsafe_paths
Log unsafe paths
2017-06-01 12:55:45 +02:00
Unrud
efb20d994f Adjust imports for isort 2017-06-01 12:45:44 +02:00
Unrud
2178ba58be Log unsafe paths 2017-06-01 12:45:44 +02:00
Unrud
8fed92dc24 Fix absolute paths in web interface when SCRIPT_NAME is used 2017-06-01 12:13:34 +02:00
Unrud
fd55bbce15 Adjust imports for isort 2017-06-01 11:54:04 +02:00
Unrud
a18874fc59 raise exception when locking the storage fails
Previously it was silently ignored, which is dangerous when multiple instances of Radicale are running.
A configuration option to disable locking was added.
2017-06-01 11:21:22 +02:00
Unrud
9585c0b048 Web interface for managing calendars and addressbooks 2017-05-31 13:18:42 +02:00
Unrud
ab9e9b2d7c Add web interface module 2017-05-31 13:18:40 +02:00
Unrud
eba6621f17 Rename backends from None to none
All other backend names are lower case.
2017-05-31 12:02:29 +02:00
Unrud
edaf21561d Don't strip SCRIPT_NAME from PATH_INFO 2017-05-31 12:01:37 +02:00
Unrud
5704b5021b PATH_INFO might not exist if it's empty 2017-05-31 12:01:35 +02:00
Unrud
f12dd31b4b Small improvements for auth tests 2017-05-31 12:01:33 +02:00
Unrud
c9664137a5 Improve error handling
* Check the configuration file for errors (check option names and basic type checking).
  * Perform basic type checking on command line arguments.
  * Only print stack traces in debug mode.
  * Include much more information in error messages (e.g. include the path of invalid files).
  * Send Bad Request to clients for invalid XML requests or iCalendar data.
  * Change the log level of some messages.
2017-05-31 11:31:54 +02:00
Unrud
c2387403dd Test the remote_user and http_x_remote user backends 2017-05-31 02:07:52 +02:00
Unrud
09bde14e50 Allow auth backends to provide login and password
This is used to implement an auth backend that takes the credentials from an HTTP header (e.g. accounts are managed by an reverse proxy)
2017-05-31 02:07:49 +02:00
Unrud
3e715a9aff Test rights management 2017-05-31 01:41:23 +02:00
Unrud
5ce2c62402 Implement rights types directly
This is faster and easier to understand.
2017-05-31 00:43:07 +02:00
Unrud
8536ffee44 Preserve empty PATH_INFO from WSGI and strip base prefix from destination 2017-05-30 22:58:57 +02:00
Unrud
11c5dfdb53 Improve handling of XML requests and responses
* Move parsing/serialization of XML requests/responses from ``xmlutils.py`` to ``__init__.py``.
  * Log XML requests/responses in pretty-printed form.
      * Previously only the responses were logged in readable form. This is useful for debugging.
      * The XML documents are only converted for pretty-printing if debugging is enabled (it's expensive)
  * Send XML responses in minimized form to clients.
  * Add **encoding** attribute to XML declaration in XML response.
  * Only decode XML requests once. (Previously they were decoded, encoded and decoded again.)
2017-05-30 09:15:51 +02:00
Unrud
f1a9cf7694 Allow already encoded answers 2017-05-30 09:02:37 +02:00
Unrud
bc49542a62 Merge pull request #587 from Unrud/testscapturelogging
Capture log messages during tests
2017-05-30 07:55:05 +02:00
Unrud
9cd9ad9a02 Only register exit function if a PID file was created
The function only deletes the PID file.
2017-05-30 07:41:50 +02:00
Unrud
01ea0a3c40 Capture log messages during tests
The log messages help finding problems with failed tests.
2017-05-30 05:44:02 +02:00
Unrud
6762fc1cad Expand user on PID path 2017-05-30 05:10:07 +02:00
Unrud
fcccb3f7af Daemonize after creation of network sockets
The original process should exit after the server is ready.
See also https://www.freedesktop.org/software/systemd/man/daemon.html#SysV%20Daemons
2017-05-30 05:10:07 +02:00
Unrud
360e88f350 Write PID file in original process
This ensures that the PID is written, when the process exists.
2017-05-30 05:10:07 +02:00
Unrud
6ade44c773 Make relative PID path absolute
The daemon changes the current directory to root.
2017-05-30 05:10:07 +02:00
Unrud
65c53df5b3 Keep original exception when PID file creation fails 2017-05-30 05:10:07 +02:00
Unrud
79bcedd4bc Refactor: Extract daemonize function 2017-05-30 05:10:07 +02:00
Guillaume Ayoub
a94a3bc7c2 Version 2.0.0 2017-05-27 18:10:46 +02:00
Guillaume Ayoub
3e0c8cf285 Update copyright years 2017-05-27 17:28:07 +02:00
Guillaume Ayoub
1825c35bc7 Merge pull request #608 from Unrud/config_file
Fail if config file is not found
2017-05-26 11:08:12 +02:00
Guillaume Ayoub
e4af3fc3cd Merge pull request #604 from Unrud/config_logging_config
Disable logging config by default
2017-05-26 11:06:27 +02:00
Unrud
577d9317d2 Strip script name from path 2017-05-26 11:02:07 +02:00
Unrud
d2a17c36ae Fail if configuration is not found
If a configuration file is passed with a command line argument and the file is not found, Radicale shows a warning and continues with the default configuration.
There is no reason for doing this, Radicale should just fail.

Instead, this PR allows passing an empty string like ``--config ""``. Radicale will use the default configuration in this case, without trying to load the configuration from the common paths. Previously you had to do specify a path that doesn't exist like ``--config /does/not/exist``, which looks a bit ugly and showed a warning message.
2017-05-24 21:01:23 +02:00
Unrud
fed974e018 Disable logging config by default
Radicale always tries to load the system-wide configuration file. To turn this off, the logging-config option has to be added to all configuration files and command line arguments. It's easier to disable it by default and only add it once to the system-wide config file.
2017-05-24 17:03:48 +02:00
Guillaume Ayoub
22d364729b Use "&" instead of "+" to test booleans 2017-05-23 17:11:28 +02:00
Guillaume Ayoub
5066e97c66 Always compare both login and password to avoid timing attacks
Related to #591.
2017-05-23 16:55:43 +02:00
Unrud
cf78a23856 Improve logging
Log failed login attempts more clearly and also log access violations of authenticated users.
2017-05-23 04:16:52 +02:00
Unrud
f2fb07fa84 Move authentication delay into __init__.py and add config
Use the delay for all backends (not only htpasswd).
Add configuration option to configure the delay.
2017-05-23 04:07:32 +02:00
Unrud
fb970246e0 Only query auth backend when a user is set 2017-05-23 04:07:32 +02:00
Unrud
c4537b1f5c Compare passwords and hashes in constant time (Fixes #591) 2017-05-23 04:07:32 +02:00
Unrud
fc309562da Repair SSHA method 2017-05-23 04:07:31 +02:00
Unrud
99bda37839 Repair auth tests, add tests for all methods and cross check
The auth tests were not doing anything because "Basic" was missing in the AUTHORIZATION header.
2017-05-23 04:07:31 +02:00
Guillaume Ayoub
dcb0638538 Version 2.0.0rc2 2017-04-19 14:10:02 +02:00
Guillaume Ayoub
059ba8dec1 Random timer to avoid timing oracles and simple bruteforce attacks
Important note: this is a security fix.
2017-04-19 13:48:30 +02:00
Guillaume Ayoub
8293ca5c1c Fix a CLI help description 2017-04-15 16:03:51 +02:00
Guillaume Ayoub
7e2406c778 Version 2.0.0rc1 2017-04-15 15:51:05 +02:00
Guillaume Ayoub
864eb5931a Merge setup.cfg and pytest.ini, fix tests 2017-04-15 10:51:00 +02:00
Guillaume Ayoub
26d8214296 Merge pull request #566 from Unrud/cleanxmlutils
Some small fixes for xmlutils.py
2017-04-15 09:24:38 +02:00
Guillaume Ayoub
b173307f55 Merge pull request #570 from Unrud/patch-5
Fix permissions for REPORT request
2017-04-15 09:20:01 +02:00
Guillaume Ayoub
d74ee7b688 Merge pull request #553 from Kozea/config
Change default values for the config
2017-04-15 09:19:32 +02:00
Unrud
4f6d2e8b58 Only search for filter that are direct children 2017-03-18 04:59:27 +01:00
Unrud
a05cca563a Stop iterating when a component is found 2017-03-18 04:58:55 +01:00
Unrud
8a98f4861d Fix permissions for REPORT request
Only read access is required.
2017-03-13 08:22:14 +01:00
Unrud
c027b68b4f PROPFIND: return all supported report methods
addressbook-multiget, addressbook-query, calendar-multiget and calendar-query were missing.
sync-collection only works for leaf collections.
2017-03-10 22:30:44 +01:00
Unrud
53a7e15833 Return empty result for unsupported report methods 2017-03-10 22:19:10 +01:00
Unrud
2f67da5750 Remove unnecessary conditions 2017-03-10 22:16:46 +01:00
Unrud
a7f12b5fac PROPFIND: only one privilege per privilege element
Before: <privilege><read /><write /></privilege>
After: <privilege><read /></privilege><privilege><write /></privilege>
2017-03-10 22:14:13 +01:00
Unrud
7d687205bd Use logger for WSGIServer
Log exception from the WSGIServer. Exceptions from socket timeouts are currently written to stderr.
2017-03-10 13:39:26 +01:00
Guillaume Ayoub
edebcf03c7 Merge pull request #558 from Unrud/collision
Check for conflicting file names
2017-03-08 15:58:48 +01:00
Guillaume Ayoub
33b27a2e71 Merge pull request #559 from Unrud/remotehostlogging
Cleanup remote host logging
2017-03-08 15:57:07 +01:00