Commit Graph

1086 Commits

Author SHA1 Message Date
Unrud
c9664137a5 Improve error handling
* Check the configuration file for errors (check option names and basic type checking).
  * Perform basic type checking on command line arguments.
  * Only print stack traces in debug mode.
  * Include much more information in error messages (e.g. include the path of invalid files).
  * Send Bad Request to clients for invalid XML requests or iCalendar data.
  * Change the log level of some messages.
2017-05-31 11:31:54 +02:00
Unrud
c2387403dd Test the remote_user and http_x_remote user backends 2017-05-31 02:07:52 +02:00
Unrud
09bde14e50 Allow auth backends to provide login and password
This is used to implement an auth backend that takes the credentials from an HTTP header (e.g. accounts are managed by an reverse proxy)
2017-05-31 02:07:49 +02:00
Unrud
3e715a9aff Test rights management 2017-05-31 01:41:23 +02:00
Unrud
5ce2c62402 Implement rights types directly
This is faster and easier to understand.
2017-05-31 00:43:07 +02:00
Unrud
8536ffee44 Preserve empty PATH_INFO from WSGI and strip base prefix from destination 2017-05-30 22:58:57 +02:00
Unrud
11c5dfdb53 Improve handling of XML requests and responses
* Move parsing/serialization of XML requests/responses from ``xmlutils.py`` to ``__init__.py``.
  * Log XML requests/responses in pretty-printed form.
      * Previously only the responses were logged in readable form. This is useful for debugging.
      * The XML documents are only converted for pretty-printing if debugging is enabled (it's expensive)
  * Send XML responses in minimized form to clients.
  * Add **encoding** attribute to XML declaration in XML response.
  * Only decode XML requests once. (Previously they were decoded, encoded and decoded again.)
2017-05-30 09:15:51 +02:00
Unrud
f1a9cf7694 Allow already encoded answers 2017-05-30 09:02:37 +02:00
Unrud
bc49542a62 Merge pull request #587 from Unrud/testscapturelogging
Capture log messages during tests
2017-05-30 07:55:05 +02:00
Unrud
9cd9ad9a02 Only register exit function if a PID file was created
The function only deletes the PID file.
2017-05-30 07:41:50 +02:00
Unrud
01ea0a3c40 Capture log messages during tests
The log messages help finding problems with failed tests.
2017-05-30 05:44:02 +02:00
Unrud
6762fc1cad Expand user on PID path 2017-05-30 05:10:07 +02:00
Unrud
fcccb3f7af Daemonize after creation of network sockets
The original process should exit after the server is ready.
See also https://www.freedesktop.org/software/systemd/man/daemon.html#SysV%20Daemons
2017-05-30 05:10:07 +02:00
Unrud
360e88f350 Write PID file in original process
This ensures that the PID is written, when the process exists.
2017-05-30 05:10:07 +02:00
Unrud
6ade44c773 Make relative PID path absolute
The daemon changes the current directory to root.
2017-05-30 05:10:07 +02:00
Unrud
65c53df5b3 Keep original exception when PID file creation fails 2017-05-30 05:10:07 +02:00
Unrud
79bcedd4bc Refactor: Extract daemonize function 2017-05-30 05:10:07 +02:00
Guillaume Ayoub
a94a3bc7c2 Version 2.0.0 2017-05-27 18:10:46 +02:00
Guillaume Ayoub
3e0c8cf285 Update copyright years 2017-05-27 17:28:07 +02:00
Guillaume Ayoub
1825c35bc7 Merge pull request #608 from Unrud/config_file
Fail if config file is not found
2017-05-26 11:08:12 +02:00
Guillaume Ayoub
e4af3fc3cd Merge pull request #604 from Unrud/config_logging_config
Disable logging config by default
2017-05-26 11:06:27 +02:00
Unrud
577d9317d2 Strip script name from path 2017-05-26 11:02:07 +02:00
Unrud
d2a17c36ae Fail if configuration is not found
If a configuration file is passed with a command line argument and the file is not found, Radicale shows a warning and continues with the default configuration.
There is no reason for doing this, Radicale should just fail.

Instead, this PR allows passing an empty string like ``--config ""``. Radicale will use the default configuration in this case, without trying to load the configuration from the common paths. Previously you had to do specify a path that doesn't exist like ``--config /does/not/exist``, which looks a bit ugly and showed a warning message.
2017-05-24 21:01:23 +02:00
Unrud
fed974e018 Disable logging config by default
Radicale always tries to load the system-wide configuration file. To turn this off, the logging-config option has to be added to all configuration files and command line arguments. It's easier to disable it by default and only add it once to the system-wide config file.
2017-05-24 17:03:48 +02:00
Guillaume Ayoub
22d364729b Use "&" instead of "+" to test booleans 2017-05-23 17:11:28 +02:00
Guillaume Ayoub
5066e97c66 Always compare both login and password to avoid timing attacks
Related to #591.
2017-05-23 16:55:43 +02:00
Unrud
cf78a23856 Improve logging
Log failed login attempts more clearly and also log access violations of authenticated users.
2017-05-23 04:16:52 +02:00
Unrud
f2fb07fa84 Move authentication delay into __init__.py and add config
Use the delay for all backends (not only htpasswd).
Add configuration option to configure the delay.
2017-05-23 04:07:32 +02:00
Unrud
fb970246e0 Only query auth backend when a user is set 2017-05-23 04:07:32 +02:00
Unrud
c4537b1f5c Compare passwords and hashes in constant time (Fixes #591) 2017-05-23 04:07:32 +02:00
Unrud
fc309562da Repair SSHA method 2017-05-23 04:07:31 +02:00
Unrud
99bda37839 Repair auth tests, add tests for all methods and cross check
The auth tests were not doing anything because "Basic" was missing in the AUTHORIZATION header.
2017-05-23 04:07:31 +02:00
Guillaume Ayoub
dcb0638538 Version 2.0.0rc2 2017-04-19 14:10:02 +02:00
Guillaume Ayoub
059ba8dec1 Random timer to avoid timing oracles and simple bruteforce attacks
Important note: this is a security fix.
2017-04-19 13:48:30 +02:00
Guillaume Ayoub
8293ca5c1c Fix a CLI help description 2017-04-15 16:03:51 +02:00
Guillaume Ayoub
7e2406c778 Version 2.0.0rc1 2017-04-15 15:51:05 +02:00
Guillaume Ayoub
864eb5931a Merge setup.cfg and pytest.ini, fix tests 2017-04-15 10:51:00 +02:00
Guillaume Ayoub
26d8214296 Merge pull request #566 from Unrud/cleanxmlutils
Some small fixes for xmlutils.py
2017-04-15 09:24:38 +02:00
Guillaume Ayoub
b173307f55 Merge pull request #570 from Unrud/patch-5
Fix permissions for REPORT request
2017-04-15 09:20:01 +02:00
Guillaume Ayoub
d74ee7b688 Merge pull request #553 from Kozea/config
Change default values for the config
2017-04-15 09:19:32 +02:00
Unrud
4f6d2e8b58 Only search for filter that are direct children 2017-03-18 04:59:27 +01:00
Unrud
a05cca563a Stop iterating when a component is found 2017-03-18 04:58:55 +01:00
Unrud
8a98f4861d Fix permissions for REPORT request
Only read access is required.
2017-03-13 08:22:14 +01:00
Unrud
c027b68b4f PROPFIND: return all supported report methods
addressbook-multiget, addressbook-query, calendar-multiget and calendar-query were missing.
sync-collection only works for leaf collections.
2017-03-10 22:30:44 +01:00
Unrud
53a7e15833 Return empty result for unsupported report methods 2017-03-10 22:19:10 +01:00
Unrud
2f67da5750 Remove unnecessary conditions 2017-03-10 22:16:46 +01:00
Unrud
a7f12b5fac PROPFIND: only one privilege per privilege element
Before: <privilege><read /><write /></privilege>
After: <privilege><read /></privilege><privilege><write /></privilege>
2017-03-10 22:14:13 +01:00
Unrud
7d687205bd Use logger for WSGIServer
Log exception from the WSGIServer. Exceptions from socket timeouts are currently written to stderr.
2017-03-10 13:39:26 +01:00
Guillaume Ayoub
edebcf03c7 Merge pull request #558 from Unrud/collision
Check for conflicting file names
2017-03-08 15:58:48 +01:00
Guillaume Ayoub
33b27a2e71 Merge pull request #559 from Unrud/remotehostlogging
Cleanup remote host logging
2017-03-08 15:57:07 +01:00
Guillaume Ayoub
9b8fc4ac14 Update and fix some config values 2017-03-08 15:50:24 +01:00
Unrud
5cd5cfe368 Use REMOTE_ADDR if REMOTE_HOST is missing
WSGIRequestHandler doesn't set REMOTE_HOST if dns lookup is disabled.
2017-03-07 20:34:37 +01:00
Unrud
69d39b47ca Also log the forwarding host for forwarded requests 2017-03-07 20:34:36 +01:00
Unrud
c104da28ce Use UNKNOWN if user agent is missing
This is much shorter and doesn't clutter the log as much.
2017-03-07 20:33:37 +01:00
Unrud
266dc608f2 Remove unnecessary if-statements 2017-03-07 20:32:23 +01:00
Unrud
c6c32945a0 Log unsafe paths when discovering collections 2017-03-07 18:24:40 +01:00
Unrud
9b27d075b6 Always close files when creating collection
If an exception occurs we rely on garbage collection to close the files.
2017-03-07 18:13:52 +01:00
Unrud
2d170bd41f Check for conflicting file names
On Windows file systems the user "TESTUS~1" can access the data of the user "testuser".
2017-03-07 18:04:51 +01:00
Guillaume Ayoub
4278cc3443 Set htpasswd encryption type to plain in tests 2017-03-04 14:25:01 +01:00
Guillaume Ayoub
78abe39002 Set authentication method when testing rights 2017-03-04 14:22:28 +01:00
Guillaume Ayoub
b47505d5bd Don't set rights management when no authentication is set 2017-03-04 14:15:46 +01:00
Guillaume Ayoub
8cc15a3d25 Change default values for the config
Fix #346.
2017-03-04 14:06:09 +01:00
Guillaume Ayoub
fc04e4543b Remove unused import 2017-02-26 16:41:59 +01:00
Guillaume Ayoub
6b683477cb Merge branch 'improved-broken-vcard-handling' of https://github.com/pbiering/Radicale into pbiering-improved-broken-vcard-handling 2017-02-26 16:35:44 +01:00
Guillaume Ayoub
5d67257555 Merge branch 'trycatch-broken-objects' of https://github.com/pbiering/Radicale into pbiering-trycatch-broken-objects 2017-02-26 16:24:41 +01:00
Guillaume Ayoub
46fcec8fe0 Cosmetics 2017-02-26 16:19:38 +01:00
Guillaume Ayoub
e6880057a2 Merge pull request #518 from pbiering/improved-access-logging
improved request logging
2017-02-26 16:17:44 +01:00
Guillaume Ayoub
9d958e8e51 Remove unused import 2017-02-26 16:13:50 +01:00
Guillaume Ayoub
04764c2af4 Merge branch 'storage' of https://github.com/Unrud/Radicale into Unrud-storage 2017-02-26 15:48:22 +01:00
Guillaume Ayoub
b9db971060 Merge branch 'fixed-propfind-handling' of https://github.com/muggenhor/Radicale into muggenhor-fixed-propfind-handling 2017-02-26 15:33:34 +01:00
Guillaume Ayoub
3213495245 Merge branch 'url' of https://github.com/Unrud/Radicale into Unrud-url 2017-02-26 14:43:59 +01:00
Guillaume Ayoub
78db0072bc Merge pull request #521 from stephane-martin/fix-support-report-set-xml-element
<report> should contain XML elements, not text
2017-02-26 14:37:38 +01:00
Guillaume Ayoub
34a82c9b69 Merge pull request #540 from twoodcock/master
remove umask(0) in daemon mode
2017-02-26 14:35:19 +01:00
Tim Woodcock
7f0d717459 remove umask(0) in daemon mode 2016-11-29 16:33:10 -08:00
Guillaume Ayoub
e55d75ce46 Set charsets in headers 2016-10-12 14:50:53 +02:00
Guillaume Ayoub
c459d32a19 Use argparse to parse command arguments
This commit also allows users to specify all the config values through
the command line.

Fix #154.
2016-10-12 14:30:18 +02:00
Yohann Rebattu
6d76cfa5aa use dash instead of underscore in options as well 2016-10-12 10:48:14 +02:00
Yohann Rebattu
fb3e733596 use dash instead of underscore 2016-10-12 10:44:13 +02:00
Yohann Rebattu
d6ee788ff5 comment… 2016-10-11 18:22:26 +02:00
Yohann Rebattu
ca2d519345 use group exclusively except for config 2016-10-11 18:21:17 +02:00
Yohann Rebattu
11710f75a8 allow config override from commandline 2016-10-11 18:17:01 +02:00
Stephane Martin
5abc8aea43 <report> should contain XML elements, not text 2016-09-20 00:27:03 +02:00
Peter Bieringer
c578470fc3 log depth also on response log line 2016-09-19 20:11:52 +02:00
Peter Bieringer
f52fa73cab check env first before using HTTP_DEPTH 2016-09-19 20:04:11 +02:00
Peter Bieringer
87061df68f add conditional logging of given depth 2016-09-19 19:59:47 +02:00
Peter Bieringer
830d6f9305 add examples of broken vcard and vevent 2016-09-18 19:36:54 +02:00
Peter Bieringer
a5e3ffa427 realign change 2016-09-18 19:30:51 +02:00
Peter Bieringer
e05206eec3 Merge branch 'improved-broken-vcard-handling' of github.com:pbiering/Radicale into improved-broken-vcard-handling 2016-09-18 19:28:15 +02:00
Peter Bieringer
af5c1582dc improved detection of broken vcards 2016-09-18 19:23:24 +02:00
Peter Bieringer
1ffdb03106 simplify patch 2016-09-17 18:16:27 +02:00
Peter Bieringer
ff02d17619 remove temporary debug statements 2016-09-17 17:44:33 +02:00
Peter Bieringer
1cfd3fd95b filter broken vcard entries before delivered, found on real life system at least 2 cards where vobject claimed about
vobject.base.ValidateError: 'VCARD components must contain at least 1 N'
2016-09-17 16:53:24 +02:00
Peter Bieringer
18181374e1 log exception for broken items 2016-09-17 16:29:40 +02:00
Peter Bieringer
d116423458 improved request logging 2016-09-17 15:35:43 +02:00
Peter Bieringer
e6ba31937a 3rd catch 2016-09-17 15:11:02 +02:00
Peter Bieringer
a57fcad270 catch 2nd case for broken object 2016-09-17 13:56:27 +02:00
Peter Bieringer
cd97aab72c try/catch of objects avoiding not serving any object in case an object has no valid contents
also log duration of how many objects are read per folder
2016-09-17 12:52:42 +02:00
Guillaume Ayoub
0c1dbc30bd Log read objects 2016-09-16 14:13:53 +02:00
Guillaume Ayoub
ecb8a99ed1 Merge branch 'master' of github.com:Kozea/radicale 2016-09-16 14:12:36 +02:00
Guillaume Ayoub
2c938ea67a Don't log when skipping .Radicale* files
Patch proposed by @pbiering.
2016-09-16 14:12:05 +02:00
Unrud
def34b1454 Fix BaseCollection.move
The update method doesn't exist.
2016-09-15 11:18:55 +02:00
Guillaume Ayoub
e95d187cbc Merge pull request #509 from Unrud/patch-2
Set password to empty string instead of None
2016-09-07 16:08:37 +02:00
Unrud
f7435814fc Repair hreferences in REPORT response
They were not extended with base_prefix.
2016-09-06 16:53:20 +02:00
Unrud
eb4b513d63 Quote hreferences
RFC 4918 states that they are URIs and RFC 3986 says that URIs must always be in percent-encoded form.
2016-09-06 16:53:20 +02:00
Unrud
83046c80c4 Let reverse proxies overwrite script name
Reverse proxies can overwrite the script name with the HTTP header field X-Script-Name.
2016-09-06 16:53:14 +02:00
Unrud
90486f33a5 Log invalid URLs in XML requests
Before the requests either failed or the invalid hreference was silently dropped.
2016-09-04 22:28:01 +02:00
Unrud
d5b8ddd71c Check that name is valid in name_from_path
Before it was possible craft XML requests, so that the storage backend got requests with invalid hrefs.
2016-09-04 22:28:01 +02:00
Unrud
139076faee Sanitize URLs from XML requests 2016-09-04 22:27:59 +02:00
Unrud
13d652b094 Remove unnecessary module prefix 2016-09-04 22:26:46 +02:00
Unrud
664fa71278 Don't double unquote request URL
"%2525" was transformed to "%" instead of "%25".
2016-09-04 22:26:46 +02:00
Unrud
dbaf58dbfe Remove base_prefix and use SCRIPT_NAME instead
This conforms with the WSGI reference (PEP 333)
2016-09-04 22:26:40 +02:00
Unrud
03fbb1e68e Don't strip {} in get method
If someone uploads a file that starts or ends with the chars {}, all REPORT requests on that collection will fail and it's impossible to delete the file.
2016-09-04 13:35:44 +02:00
Unrud
6df54bf88a Log name of faulty component
If vobject can't parse a component it raises an exception, but the filename of that component is missing in the logs.
2016-09-04 13:23:01 +02:00
Unrud
5ccfe16372 Remove Collection.has
It's the same as BaseCollection.has
2016-09-04 13:21:57 +02:00
Unrud
de09f6689a Only relevant files for last_modified calculation
Leftovers from failed transactions etc. should not change that property.
2016-09-04 13:16:42 +02:00
Unrud
cd9f789294 Name variables for files f
fd sounds more like file descriptions.
prop doesn't sound like a file at all.
2016-09-04 13:14:51 +02:00
Unrud
f5650df5f7 Remove checks for existence of collection
They are unnecessary since the discover methods stopped returning collections that actually don't exist.
2016-09-04 13:13:35 +02:00
Unrud
e7d8b4816c Duplicate code: Use list and get methods 2016-09-04 13:12:55 +02:00
Unrud
5dbf9df876 Add missing checks for safe fileystem components
Currently it's not possible to exploit these.
2016-09-04 13:09:10 +02:00
Unrud
dc501d5dc5 Refactor/Duplicate code: Extract _fsync method 2016-09-04 13:08:12 +02:00
Unrud
77e9ca1252 Remove EtagMismatchError
Etags are not checked in storage anymore and this is unused.
2016-09-04 13:06:09 +02:00
Unrud
a12ef69129 Secure is_safe_filesystem_path_component
On Windows 1/2 would be a safe filesystem path component, but it's not safe to pass it to path_to_filesystem.
Currently only the get method can be called with a href like that and it checked for that.
This just moves the check into the is_safe_filesystem_path_component function.
2016-09-04 12:55:28 +02:00
Unrud
a4a6a62643 Duplicate code: Use is_safe_path_component 2016-09-04 12:53:07 +02:00
Unrud
8d5f2ded42 Describe encoding of Etag 2016-09-04 12:52:29 +02:00
Unrud
d371179487 thread and level in simple log formatter
The log messages are a mess without any allocation to threads.
2016-09-04 11:47:27 +02:00
Unrud
fe5daf801a Set password to empty string instead of None
Prevent exception in auth module.
2016-09-03 10:01:52 +02:00
Guillaume Ayoub
b85fc5bed6 Merge pull request #507 from Unrud/auth
Repair and test authentication
2016-09-02 18:36:31 +02:00
Unrud
9e27d4e2a8 Emulate fullmatch with match
re.fullmatch was introduced in Python 3.4
2016-09-02 15:06:32 +02:00
Unrud
b25a601e28 Test custom header 2016-09-02 14:52:45 +02:00
Unrud
11df2f1184 Test authentication
Test for 2a9f37defb
2016-09-02 14:42:22 +02:00
Unrud
2a9f37defb Repair authentication 2016-09-02 14:41:31 +02:00
Guillaume Ayoub
88d558f9fa Remove useless nested tuples 2016-09-02 11:05:35 +02:00
Guillaume Ayoub
68e1e9dfb2 Don't use mutables in constants or parameters 2016-09-02 11:04:29 +02:00
Unrud
20b1480399 Make copy of headers before mutating 2016-09-02 04:23:47 +02:00
Unrud
f7e995f9f6 Move encoding of answer into response function
Fix #505
2016-09-02 04:10:11 +02:00
Unrud
3df367aad2 Test REPORT on item
Test for 90e4655a44
2016-09-01 06:17:18 +02:00
Unrud
9d2743d797 Test authorization 2016-09-01 06:15:31 +02:00
Guillaume Ayoub
e25373fa85 Merge pull request #501 from Unrud/httperrors
HTTP error messages
2016-08-31 14:49:19 +02:00
Guillaume Ayoub
8d92b371e4 Merge pull request #502 from Unrud/patch-40
Plain text for GET / instead of malformed HTML
2016-08-31 14:42:59 +02:00
Guillaume Ayoub
db681da08e Merge pull request #500 from Unrud/loginuser
Map logins to internal users in Auth module
2016-08-31 14:40:08 +02:00
Unrud
f875bcd892 Plain text for GET / instead of malformed HTML
It's basically the same in browsers and looks nicer in tools that don't support HTML rendering (like curl).
2016-08-31 02:01:18 +02:00
Unrud
ff2b8f6e5c Use NOT_FOUND instead of GONE
Thunderbird doesn't recognize the status code correctly and shows an synchronization error.
2016-08-31 01:54:31 +02:00
Unrud
1ea9b1dca9 Return error for GET requests on directories
Radicale doesn't support directory listings.
2016-08-31 00:45:14 +02:00
Unrud
4ac2e68f5c Add HTTP error messages
Browsers just show a blank page if an error occurs. You have to open the developer tools to see the HTTP status code. E.g. a user wants to download a calendar in the browser and the URL is wrong.

Some tools like curl don't show any indication of an error.
2016-08-31 00:41:08 +02:00
Unrud
689e5c9dd5 Map logins to internal users in Auth module
This makes it possible to implement #349 as a Auth module. Another use case would be to encode usernames that contain characters unsupported by the file system.
2016-08-30 23:13:33 +02:00
Unrud
e782808a14 Remove empty line in log
Separating requests by an empty line doesn't work any longer with parallel (overlapping) requests.
2016-08-30 14:04:16 +02:00
Markus Unterwaditzer
90e4655a44 Don't return <item>/ for events 2016-08-29 17:29:46 +02:00
Guillaume Ayoub
947c57312c Fix a variable name
Fix #494
2016-08-29 12:09:17 +02:00
Guillaume Ayoub
34ad1b9073 Remove useless import 2016-08-29 12:07:58 +02:00
Guillaume Ayoub
fd1742fcea Fix typo 2016-08-29 12:07:30 +02:00