<h1>Test Coverage<aclass="headerlink"href="#test-coverage"title="Link to this heading">¶</a></h1>
<p>Writing unit tests for your application lets you check that the code
you wrote works the way you expect. Flask provides a test client that
simulates requests to the application and returns the response data.</p>
<p>You should test as much of your code as possible. Code in functions only
runs when the function is called, and code in branches, such as <codeclass="docutils literal notranslate"><spanclass="pre">if</span></code>
blocks, only runs when the condition is met. You want to make sure that
each function is tested with data that covers each branch.</p>
<p>The closer you get to 100% coverage, the more comfortable you can be
that making a change won’t unexpectedly change other behavior. However,
100% coverage doesn’t guarantee that your application doesn’t have bugs.
In particular, it doesn’t test how the user interacts with the
application in the browser. Despite this, test coverage is an important
tool to use during development.</p>
<divclass="admonition note">
<pclass="admonition-title">Note</p>
<p>This is being introduced late in the tutorial, but in your future
projects you should test as you develop.</p>
</div>
<p>You’ll use <aclass="reference external"href="https://pytest.readthedocs.io/">pytest</a> and <aclass="reference external"href="https://coverage.readthedocs.io/">coverage</a> to test and measure your code.
<divclass="code-block-caption"><spanclass="caption-text"><codeclass="docutils literal notranslate"><spanclass="pre">tests/data.sql</span></code></span><aclass="headerlink"href="#id1"title="Link to this code">¶</a></div>
<p>The <codeclass="docutils literal notranslate"><spanclass="pre">app</span></code> fixture will call the factory and pass <codeclass="docutils literal notranslate"><spanclass="pre">test_config</span></code> to
configure the application and database for testing instead of using your
<divclass="code-block-caption"><spanclass="caption-text"><codeclass="docutils literal notranslate"><spanclass="pre">tests/conftest.py</span></code></span><aclass="headerlink"href="#id2"title="Link to this code">¶</a></div>
<p><aclass="reference external"href="https://docs.python.org/3/library/tempfile.html#tempfile.mkstemp"title="(in Python v3.13)"><codeclass="xref py py-func docutils literal notranslate"><spanclass="pre">tempfile.mkstemp()</span></code></a> creates and opens a temporary file, returning
the file descriptor and the path to it. The <codeclass="docutils literal notranslate"><spanclass="pre">DATABASE</span></code> path is
overridden so it points to this temporary path instead of the instance
folder. After setting the path, the database tables are created and the
test data is inserted. After the test is over, the temporary file is
closed and removed.</p>
<p><aclass="reference internal"href="../config.html#TESTING"title="TESTING"><codeclass="xref py py-data docutils literal notranslate"><spanclass="pre">TESTING</span></code></a> tells Flask that the app is in test mode. Flask changes
some internal behavior so it’s easier to test, and other extensions can
also use the flag to make testing them easier.</p>
<aclass="reference internal"href="../api.html#flask.Flask.test_client"title="flask.Flask.test_client"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">app.test_client()</span></code></a> with the application
object created by the <codeclass="docutils literal notranslate"><spanclass="pre">app</span></code> fixture. Tests will use the client to make
requests to the application without running the server.</p>
<p>The <codeclass="docutils literal notranslate"><spanclass="pre">runner</span></code> fixture is similar to <codeclass="docutils literal notranslate"><spanclass="pre">client</span></code>.
<aclass="reference internal"href="../api.html#flask.Flask.test_cli_runner"title="flask.Flask.test_cli_runner"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">app.test_cli_runner()</span></code></a> creates a runner
that can call the Click commands registered with the application.</p>
<p>Pytest uses fixtures by matching their function names with the names
of arguments in the test functions. For example, the <codeclass="docutils literal notranslate"><spanclass="pre">test_hello</span></code>
function you’ll write next takes a <codeclass="docutils literal notranslate"><spanclass="pre">client</span></code> argument. Pytest matches
that with the <codeclass="docutils literal notranslate"><spanclass="pre">client</span></code> fixture function, calls it, and passes the
returned value to the test function.</p>
</section>
<sectionid="factory">
<h2>Factory<aclass="headerlink"href="#factory"title="Link to this heading">¶</a></h2>
<p>There’s not much to test about the factory itself. Most of the code will
be executed for each test already, so if something fails the other tests
will notice.</p>
<p>The only behavior that can change is passing test config. If config is
not passed, there should be some default configuration, otherwise the
<divclass="code-block-caption"><spanclass="caption-text"><codeclass="docutils literal notranslate"><spanclass="pre">tests/test_factory.py</span></code></span><aclass="headerlink"href="#id3"title="Link to this code">¶</a></div>
<divclass="code-block-caption"><spanclass="caption-text"><codeclass="docutils literal notranslate"><spanclass="pre">tests/test_db.py</span></code></span><aclass="headerlink"href="#id4"title="Link to this code">¶</a></div>
<p>The <codeclass="docutils literal notranslate"><spanclass="pre">init-db</span></code> command should call the <codeclass="docutils literal notranslate"><spanclass="pre">init_db</span></code> function and output
<divclass="code-block-caption"><spanclass="caption-text"><codeclass="docutils literal notranslate"><spanclass="pre">tests/test_db.py</span></code></span><aclass="headerlink"href="#id5"title="Link to this code">¶</a></div>
<p>This test uses Pytest’s <codeclass="docutils literal notranslate"><spanclass="pre">monkeypatch</span></code> fixture to replace the
<codeclass="docutils literal notranslate"><spanclass="pre">init_db</span></code> function with one that records that it’s been called. The
<codeclass="docutils literal notranslate"><spanclass="pre">runner</span></code> fixture you wrote above is used to call the <codeclass="docutils literal notranslate"><spanclass="pre">init-db</span></code>
command by name.</p>
</section>
<sectionid="authentication">
<h2>Authentication<aclass="headerlink"href="#authentication"title="Link to this heading">¶</a></h2>
<p>For most of the views, a user needs to be logged in. The easiest way to
do this in tests is to make a <codeclass="docutils literal notranslate"><spanclass="pre">POST</span></code> request to the <codeclass="docutils literal notranslate"><spanclass="pre">login</span></code> view
with the client. Rather than writing that out every time, you can write
a class with methods to do that, and use a fixture to pass it the client
<divclass="code-block-caption"><spanclass="caption-text"><codeclass="docutils literal notranslate"><spanclass="pre">tests/conftest.py</span></code></span><aclass="headerlink"href="#id6"title="Link to this code">¶</a></div>
<p>With the <codeclass="docutils literal notranslate"><spanclass="pre">auth</span></code> fixture, you can call <codeclass="docutils literal notranslate"><spanclass="pre">auth.login()</span></code> in a test to
log in as the <codeclass="docutils literal notranslate"><spanclass="pre">test</span></code> user, which was inserted as part of the test
data in the <codeclass="docutils literal notranslate"><spanclass="pre">app</span></code> fixture.</p>
<p>The <codeclass="docutils literal notranslate"><spanclass="pre">register</span></code> view should render successfully on <codeclass="docutils literal notranslate"><spanclass="pre">GET</span></code>. On <codeclass="docutils literal notranslate"><spanclass="pre">POST</span></code>
with valid form data, it should redirect to the login URL and the user’s
data should be in the database. Invalid data should display error
<divclass="code-block-caption"><spanclass="caption-text"><codeclass="docutils literal notranslate"><spanclass="pre">tests/test_auth.py</span></code></span><aclass="headerlink"href="#id7"title="Link to this code">¶</a></div>
<spanclass="p">(</span><spanclass="s1">''</span><spanclass="p">,</span><spanclass="s1">''</span><spanclass="p">,</span><spanclass="sa">b</span><spanclass="s1">'Username is required.'</span><spanclass="p">),</span>
<spanclass="p">(</span><spanclass="s1">'a'</span><spanclass="p">,</span><spanclass="s1">''</span><spanclass="p">,</span><spanclass="sa">b</span><spanclass="s1">'Password is required.'</span><spanclass="p">),</span>
<p><aclass="reference external"href="https://werkzeug.palletsprojects.com/en/stable/test/#werkzeug.test.Client.get"title="(in Werkzeug v3.1.x)"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">client.get()</span></code></a> makes a <codeclass="docutils literal notranslate"><spanclass="pre">GET</span></code> request
and returns the <aclass="reference internal"href="../api.html#flask.Response"title="flask.Response"><codeclass="xref py py-class docutils literal notranslate"><spanclass="pre">Response</span></code></a> object returned by Flask. Similarly,
<aclass="reference external"href="https://werkzeug.palletsprojects.com/en/stable/test/#werkzeug.test.Client.post"title="(in Werkzeug v3.1.x)"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">client.post()</span></code></a> makes a <codeclass="docutils literal notranslate"><spanclass="pre">POST</span></code>
request, converting the <codeclass="docutils literal notranslate"><spanclass="pre">data</span></code> dict into form data.</p>
<p>To test that the page renders successfully, a simple request is made and
checked for a <codeclass="docutils literal notranslate"><spanclass="pre">200</span><spanclass="pre">OK</span></code><aclass="reference internal"href="../api.html#flask.Response.status_code"title="flask.Response.status_code"><codeclass="xref py py-attr docutils literal notranslate"><spanclass="pre">status_code</span></code></a>. If
rendering failed, Flask would return a <codeclass="docutils literal notranslate"><spanclass="pre">500</span><spanclass="pre">Internal</span><spanclass="pre">Server</span><spanclass="pre">Error</span></code>
code.</p>
<p><codeclass="xref py py-attr docutils literal notranslate"><spanclass="pre">headers</span></code> will have a <codeclass="docutils literal notranslate"><spanclass="pre">Location</span></code> header with the login
URL when the register view redirects to the login view.</p>
<p><aclass="reference internal"href="../api.html#flask.Response.data"title="flask.Response.data"><codeclass="xref py py-attr docutils literal notranslate"><spanclass="pre">data</span></code></a> contains the body of the response as bytes. If
you expect a certain value to render on the page, check that it’s in
<codeclass="docutils literal notranslate"><spanclass="pre">data</span></code>. Bytes must be compared to bytes. If you want to compare text,
use <aclass="reference external"href="https://werkzeug.palletsprojects.com/en/stable/wrappers/#werkzeug.wrappers.Response.get_data"title="(in Werkzeug v3.1.x)"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">get_data(as_text=True)</span></code></a>
instead.</p>
<p><codeclass="docutils literal notranslate"><spanclass="pre">pytest.mark.parametrize</span></code> tells Pytest to run the same test function
with different arguments. You use it here to test different invalid
input and error messages without writing the same code three times.</p>
<p>The tests for the <codeclass="docutils literal notranslate"><spanclass="pre">login</span></code> view are very similar to those for
<codeclass="docutils literal notranslate"><spanclass="pre">register</span></code>. Rather than testing the data in the database,
<aclass="reference internal"href="../api.html#flask.session"title="flask.session"><codeclass="xref py py-data docutils literal notranslate"><spanclass="pre">session</span></code></a> should have <codeclass="docutils literal notranslate"><spanclass="pre">user_id</span></code> set after logging in.</p>
<divclass="code-block-caption"><spanclass="caption-text"><codeclass="docutils literal notranslate"><spanclass="pre">tests/test_auth.py</span></code></span><aclass="headerlink"href="#id8"title="Link to this code">¶</a></div>
<p>Using <codeclass="docutils literal notranslate"><spanclass="pre">client</span></code> in a <codeclass="docutils literal notranslate"><spanclass="pre">with</span></code> block allows accessing context variables
such as <aclass="reference internal"href="../api.html#flask.session"title="flask.session"><codeclass="xref py py-data docutils literal notranslate"><spanclass="pre">session</span></code></a> after the response is returned. Normally,
accessing <codeclass="docutils literal notranslate"><spanclass="pre">session</span></code> outside of a request would raise an error.</p>
<p>Testing <codeclass="docutils literal notranslate"><spanclass="pre">logout</span></code> is the opposite of <codeclass="docutils literal notranslate"><spanclass="pre">login</span></code>. <aclass="reference internal"href="../api.html#flask.session"title="flask.session"><codeclass="xref py py-data docutils literal notranslate"><spanclass="pre">session</span></code></a> should
not contain <codeclass="docutils literal notranslate"><spanclass="pre">user_id</span></code> after logging out.</p>
<divclass="code-block-caption"><spanclass="caption-text"><codeclass="docutils literal notranslate"><spanclass="pre">tests/test_auth.py</span></code></span><aclass="headerlink"href="#id9"title="Link to this code">¶</a></div>
<divclass="code-block-caption"><spanclass="caption-text"><codeclass="docutils literal notranslate"><spanclass="pre">tests/test_blog.py</span></code></span><aclass="headerlink"href="#id10"title="Link to this code">¶</a></div>
<spanclass="k">assert</span><spanclass="sa">b</span><spanclass="s1">'by test on 2018-01-01'</span><spanclass="ow">in</span><spanclass="n">response</span><spanclass="o">.</span><spanclass="n">data</span>
<p>A user must be logged in to access the <codeclass="docutils literal notranslate"><spanclass="pre">create</span></code>, <codeclass="docutils literal notranslate"><spanclass="pre">update</span></code>, and
<codeclass="docutils literal notranslate"><spanclass="pre">delete</span></code> views. The logged in user must be the author of the post to
access <codeclass="docutils literal notranslate"><spanclass="pre">update</span></code> and <codeclass="docutils literal notranslate"><spanclass="pre">delete</span></code>, otherwise a <codeclass="docutils literal notranslate"><spanclass="pre">403</span><spanclass="pre">Forbidden</span></code> status
is returned. If a <codeclass="docutils literal notranslate"><spanclass="pre">post</span></code> with the given <codeclass="docutils literal notranslate"><spanclass="pre">id</span></code> doesn’t exist,
<codeclass="docutils literal notranslate"><spanclass="pre">update</span></code> and <codeclass="docutils literal notranslate"><spanclass="pre">delete</span></code> should return <codeclass="docutils literal notranslate"><spanclass="pre">404</span><spanclass="pre">Not</span><spanclass="pre">Found</span></code>.</p>
<divclass="code-block-caption"><spanclass="caption-text"><codeclass="docutils literal notranslate"><spanclass="pre">tests/test_blog.py</span></code></span><aclass="headerlink"href="#id11"title="Link to this code">¶</a></div>
<spanclass="n">db</span><spanclass="o">.</span><spanclass="n">execute</span><spanclass="p">(</span><spanclass="s1">'UPDATE post SET author_id = 2 WHERE id = 1'</span><spanclass="p">)</span>
<p>The <codeclass="docutils literal notranslate"><spanclass="pre">create</span></code> and <codeclass="docutils literal notranslate"><spanclass="pre">update</span></code> views should render and return a
<codeclass="docutils literal notranslate"><spanclass="pre">200</span><spanclass="pre">OK</span></code> status for a <codeclass="docutils literal notranslate"><spanclass="pre">GET</span></code> request. When valid data is sent in a
<codeclass="docutils literal notranslate"><spanclass="pre">POST</span></code> request, <codeclass="docutils literal notranslate"><spanclass="pre">create</span></code> should insert the new post data into the
database, and <codeclass="docutils literal notranslate"><spanclass="pre">update</span></code> should modify the existing data. Both pages
<divclass="code-block-caption"><spanclass="caption-text"><codeclass="docutils literal notranslate"><spanclass="pre">tests/test_blog.py</span></code></span><aclass="headerlink"href="#id12"title="Link to this code">¶</a></div>
<spanclass="n">count</span><spanclass="o">=</span><spanclass="n">db</span><spanclass="o">.</span><spanclass="n">execute</span><spanclass="p">(</span><spanclass="s1">'SELECT COUNT(id) FROM post'</span><spanclass="p">)</span><spanclass="o">.</span><spanclass="n">fetchone</span><spanclass="p">()[</span><spanclass="mi">0</span><spanclass="p">]</span>
<spanclass="n">post</span><spanclass="o">=</span><spanclass="n">db</span><spanclass="o">.</span><spanclass="n">execute</span><spanclass="p">(</span><spanclass="s1">'SELECT * FROM post WHERE id = 1'</span><spanclass="p">)</span><spanclass="o">.</span><spanclass="n">fetchone</span><spanclass="p">()</span>
<spanclass="k">assert</span><spanclass="sa">b</span><spanclass="s1">'Title is required.'</span><spanclass="ow">in</span><spanclass="n">response</span><spanclass="o">.</span><spanclass="n">data</span>
</pre></div>
</div>
</div>
<p>The <codeclass="docutils literal notranslate"><spanclass="pre">delete</span></code> view should redirect to the index URL and the post should
<divclass="code-block-caption"><spanclass="caption-text"><codeclass="docutils literal notranslate"><spanclass="pre">tests/test_blog.py</span></code></span><aclass="headerlink"href="#id13"title="Link to this code">¶</a></div>
<spanclass="n">post</span><spanclass="o">=</span><spanclass="n">db</span><spanclass="o">.</span><spanclass="n">execute</span><spanclass="p">(</span><spanclass="s1">'SELECT * FROM post WHERE id = 1'</span><spanclass="p">)</span><spanclass="o">.</span><spanclass="n">fetchone</span><spanclass="p">()</span>
<divclass="code-block-caption"><spanclass="caption-text"><codeclass="docutils literal notranslate"><spanclass="pre">pyproject.toml</span></code></span><aclass="headerlink"href="#id14"title="Link to this code">¶</a></div>
========================= test session starts ==========================
platform linux -- Python 3.6.4, pytest-3.5.0, py-1.5.3, pluggy-0.6.0
rootdir: /home/user/Projects/flask-tutorial
collected 23 items
tests/test_auth.py ........ [ 34%]
tests/test_blog.py ............ [ 86%]
tests/test_db.py .. [ 95%]
tests/test_factory.py .. [100%]
====================== 24 passed in 0.64 seconds =======================
</pre></div>
</div>
<p>If any tests fail, pytest will show the error that was raised. You can
run <codeclass="docutils literal notranslate"><spanclass="pre">pytest</span><spanclass="pre">-v</span></code> to get a list of each test function rather than dots.</p>
<p>To measure the code coverage of your tests, use the <codeclass="docutils literal notranslate"><spanclass="pre">coverage</span></code> command
to run pytest instead of running it directly.</p>
<divclass="highlight-none notranslate"><divclass="highlight"><pre><span></span>$ coverage run -m pytest
</pre></div>
</div>
<p>You can either view a simple coverage report in the terminal:</p>
