maya-chen/flask
1
0
Fork 0
forked from orbit-oss/flask
Code Pull requests Activity

Add support for serializing top-level arrays to JSON

Browse source 
Fix #170, #248, #510, #673, #1177
This commit is contained in:
Jeff Widman 2015-12-30 03:11:53 -08:00
parent f17e6061fc
commit daceb3e3a0
5 changed files with 114 additions and 115 deletions
Download patch file Download diff file Expand all files Collapse all files

3
AUTHORS
View file

@ -15,9 +15,11 @@ Patches and Suggestions
- Chris Grindstaff
- Christopher Grebs
- Daniel Neuhäuser
- David Lord @davidism
- Edmond Burnett
- Florent Xicluna
- Georg Brandl
- Jeff Widman @jeffwidman
- Justin Quick
- Kenneth Reitz
- Keyan Pishdadian
@ -32,4 +34,3 @@ Patches and Suggestions
- Stephane Wirtel
- Thomas Schranz
- Zhao Xiaohong
- David Lord @davidism

3
CHANGES
View file

@ -8,6 +8,9 @@ Version 1.0
(release date to be announced, codename to be selected)
- Added support to serializing top-level arrays to :func:`flask.jsonify`. This
introduces a security risk in ancient browsers. See
:ref:`json_security` for details.
- Added before_render_template signal.
- Added `**kwargs` to :meth:`flask.Test.test_client` to support passing
additional keyword arguments to the constructor of

85
docs/security.rst
View file

@ -95,81 +95,12 @@ the form validation framework, which does not exist in Flask.
JSON Security
-------------
.. admonition:: ECMAScript 5 Changes
In Flask 0.10 and lower, :func:`~flask.jsonify` did not serialize top-level
arrays to JSON. This was because of a security vulnerability in ECMAScript 4.
Starting with ECMAScript 5 the behavior of literals changed. Now they
are not constructed with the constructor of ``Array`` and others, but
with the builtin constructor of ``Array`` which closes this particular
attack vector.
JSON itself is a high-level serialization format, so there is barely
anything that could cause security problems, right? You can't declare
recursive structures that could cause problems and the only thing that
could possibly break are very large responses that can cause some kind of
denial of service at the receiver's side.
However there is a catch. Due to how browsers work the CSRF issue comes
up with JSON unfortunately. Fortunately there is also a weird part of the
JavaScript specification that can be used to solve that problem easily and
Flask is kinda doing that for you by preventing you from doing dangerous
stuff. Unfortunately that protection is only there for
:func:`~flask.jsonify` so you are still at risk when using other ways to
generate JSON.
So what is the issue and how to avoid it? The problem are arrays at
top-level in JSON. Imagine you send the following data out in a JSON
request. Say that's exporting the names and email addresses of all your
friends for a part of the user interface that is written in JavaScript.
Not very uncommon:
.. sourcecode:: javascript
[
{"username": "admin",
"email": "admin@localhost"}
]
And it is doing that of course only as long as you are logged in and only
for you. And it is doing that for all ``GET`` requests to a certain URL,
say the URL for that request is
``http://example.com/api/get_friends.json``.
So now what happens if a clever hacker is embedding this to his website
and social engineers a victim to visiting his site:
.. sourcecode:: html
<script type=text/javascript>
var captured = [];
var oldArray = Array;
function Array() {
var obj = this, id = 0, capture = function(value) {
obj.__defineSetter__(id++, capture);
if (value)
captured.push(value);
};
capture();
}
</script>
<script type=text/javascript
src=http://example.com/api/get_friends.json></script>
<script type=text/javascript>
Array = oldArray;
// now we have all the data in the captured array.
</script>
If you know a bit of JavaScript internals you might know that it's
possible to patch constructors and register callbacks for setters. An
attacker can use this (like above) to get all the data you exported in
your JSON file. The browser will totally ignore the :mimetype:`application/json`
mimetype if :mimetype:`text/javascript` is defined as content type in the script
tag and evaluate that as JavaScript. Because top-level array elements are
allowed (albeit useless) and we hooked in our own constructor, after that
page loaded the data from the JSON response is in the `captured` array.
Because it is a syntax error in JavaScript to have an object literal
(``{...}``) toplevel an attacker could not just do a request to an
external URL with the script tag to load up the data. So what Flask does
is to only allow objects as toplevel elements when using
:func:`~flask.jsonify`. Make sure to do the same when using an ordinary
JSON generate function.
ECMAScript 5 closed this vulnerability, so only extremely old browsers are
still vulnerable. All of these browsers have `other more serious
vulnerabilities
<https://github.com/mitsuhiko/flask/issues/248#issuecomment-59934857>`_, so
this behavior was changed and :func:`~flask.jsonify` now supports serializing
arrays.

40
flask/json.py
View file

@ -199,10 +199,22 @@ def htmlsafe_dump(obj, fp, **kwargs):
def jsonify(*args, **kwargs):
"""Creates a :class:`~flask.Response` with the JSON representation of
the given arguments with an :mimetype:`application/json` mimetype. The
arguments to this function are the same as to the :class:`dict`
constructor.
"""This function wraps :func:`dumps` to add a few enhancements that make
life easier. It turns the JSON output into a :class:`~flask.Response`
object with the :mimetype:`application/json` mimetype. For convenience, it
also converts multiple arguments into an array or multiple keyword arguments
into a dict. This means that both ``jsonify(1,2,3)`` and
``jsonify([1,2,3])`` serialize to ``[1,2,3]``.
For clarity, the JSON serialization behavior has the following differences
from :func:`dumps`:
1. Single argument: Passed straight through to :func:`dumps`.
2. Multiple arguments: Converted to an array before being passed to
:func:`dumps`.
3. Multiple keyword arguments: Converted to a dict before being passed to
:func:`dumps`.
4. Both args and kwargs: Behavior undefined and will throw an exception.
Example usage::
@ -222,8 +234,10 @@ def jsonify(*args, **kwargs):
"id": 42
}
For security reasons only objects are supported toplevel. For more
information about this, have a look at :ref:`json-security`.
.. versionchanged:: 1.0
Added support for serializing top-level arrays. This introduces a
security risk in ancient browsers. See :ref:`json_security` for details.
This function's response will be pretty printed if it was not requested
with ``X-Requested-With: XMLHttpRequest`` to simplify debugging unless
@ -242,11 +256,21 @@ def jsonify(*args, **kwargs):
indent = 2
separators = (', ', ': ')
if args and kwargs:
raise TypeError(
"jsonify() behavior undefined when passed both args and kwargs"
)
elif len(args) == 1: # single args are passed directly to dumps()
data = args[0]
elif args: # convert multiple args into an array
data = list(args)
else: # convert kwargs to a dict
data = dict(kwargs)
# Note that we add '\n' to end of response
# (see https://github.com/mitsuhiko/flask/pull/1262)
rv = current_app.response_class(
(dumps(dict(*args, **kwargs), indent=indent, separators=separators),
'\n'),
(dumps(data, indent=indent, separators=separators), '\n'),
mimetype='application/json')
return rv

98
tests/test_helpers.py
View file

@ -31,24 +31,6 @@ def has_encoding(name):
class TestJSON(object):
def test_jsonify_date_types(self):
"""Test jsonify with datetime.date and datetime.datetime types."""
test_dates = (
datetime.datetime(1973, 3, 11, 6, 30, 45),
datetime.date(1975, 1, 5)
)
app = flask.Flask(__name__)
c = app.test_client()
for i, d in enumerate(test_dates):
url = '/datetest{0}'.format(i)
app.add_url_rule(url, str(i), lambda val=d: flask.jsonify(x=val))
rv = c.get(url)
assert rv.mimetype == 'application/json'
assert flask.json.loads(rv.data)['x'] == http_date(d.timetuple())
def test_post_empty_json_adds_exception_to_response_content_in_debug(self):
app = flask.Flask(__name__)
app.config['DEBUG'] = True
@ -103,8 +85,41 @@ class TestJSON(object):
content_type='application/json; charset=iso-8859-15')
assert resp.data == u'Hällo Wörld'.encode('utf-8')
def test_jsonify(self):
d = dict(a=23, b=42, c=[1, 2, 3])
def test_json_as_unicode(self):
app = flask.Flask(__name__)
app.config['JSON_AS_ASCII'] = True
with app.app_context():
rv = flask.json.dumps(u'\N{SNOWMAN}')
assert rv == '"\\u2603"'
app.config['JSON_AS_ASCII'] = False
with app.app_context():
rv = flask.json.dumps(u'\N{SNOWMAN}')
assert rv == u'"\u2603"'
def test_jsonify_basic_types(self):
"""Test jsonify with basic types."""
# Should be able to use pytest parametrize on this, but I couldn't
# figure out the correct syntax
# https://pytest.org/latest/parametrize.html#pytest-mark-parametrize-parametrizing-test-functions
test_data = (0, 1, 23, 3.14, 's', "longer string", True, False,)
app = flask.Flask(__name__)
c = app.test_client()
for i, d in enumerate(test_data):
url = '/jsonify_basic_types{0}'.format(i)
app.add_url_rule(url, str(i), lambda x=d: flask.jsonify(x))
rv = c.get(url)
assert rv.mimetype == 'application/json'
assert flask.json.loads(rv.data) == d
def test_jsonify_dicts(self):
"""Test jsonify with dicts and kwargs unpacking."""
d = dict(
a=0, b=23, c=3.14, d='t', e='Hi', f=True, g=False,
h=['test list', 10, False],
i={'test':'dict'}
)
app = flask.Flask(__name__)
@app.route('/kw')
def return_kwargs():
@ -118,18 +133,43 @@ class TestJSON(object):
assert rv.mimetype == 'application/json'
assert flask.json.loads(rv.data) == d
def test_json_as_unicode(self):
def test_jsonify_arrays(self):
"""Test jsonify of lists and args unpacking."""
l = [
0, 42, 3.14, 't', 'hello', True, False,
['test list', 2, False],
{'test':'dict'}
]
app = flask.Flask(__name__)
@app.route('/args_unpack')
def return_args_unpack():
return flask.jsonify(*l)
@app.route('/array')
def return_array():
return flask.jsonify(l)
c = app.test_client()
for url in '/args_unpack', '/array':
rv = c.get(url)
assert rv.mimetype == 'application/json'
assert flask.json.loads(rv.data) == l
app.config['JSON_AS_ASCII'] = True
with app.app_context():
rv = flask.json.dumps(u'\N{SNOWMAN}')
assert rv == '"\\u2603"'
def test_jsonify_date_types(self):
"""Test jsonify with datetime.date and datetime.datetime types."""
app.config['JSON_AS_ASCII'] = False
with app.app_context():
rv = flask.json.dumps(u'\N{SNOWMAN}')
assert rv == u'"\u2603"'
test_dates = (
datetime.datetime(1973, 3, 11, 6, 30, 45),
datetime.date(1975, 1, 5)
)
app = flask.Flask(__name__)
c = app.test_client()
for i, d in enumerate(test_dates):
url = '/datetest{0}'.format(i)
app.add_url_rule(url, str(i), lambda val=d: flask.jsonify(x=val))
rv = c.get(url)
assert rv.mimetype == 'application/json'
assert flask.json.loads(rv.data)['x'] == http_date(d.timetuple())
def test_json_attr(self):
app = flask.Flask(__name__)