Performance

Chapter 16. Performance

Nobody likes slow applications. Long waits for pages to load frustrate users, so it is important to detect and correct performance problems as soon as they appear. In this chapter, two important performance aspects of web applications are considered.

Logging Slow Database Performance

When application performance slowly degenerates with time, it is likely due to slow database queries, which get worse as the size of the database grows. Optimizing database queries can be as simple as adding more indexes or as complex as adding a cache between the application and the database. The explain statement, available in most database query languages, shows the steps the database takes to execute a given query, often exposing inefficiencies in database or index design.

But before starting to optimize queries, it is necessary to determine which queries are the ones that are worth optimizing. During a typical request several database queries may be issued, so it is often hard to identify which of all the queries are the slow ones. Flask-SQLAlchemy has an option to record statistics about database queries issued during a request. In Example 16-1 you can see how this feature can be used to log queries that are slower than a configured threshold.

Example 16-1. app/main/views.py: reporting slow database queries

This functionality is attached to an after_app_request handler, which works in a similar way to the before_app_request handler but is invoked after the view function that handles the request returns. Flask passes the response object to the after_app_request handler in case it needs to be modified.

In this case, the after_app_request handler does not modify the response; it just gets the query timings recorded by Flask-SQLAlchemy and then logs the slow ones to the application logger that Flask sets up at app.logger, before returning the response, which will then be sent to the client.

The get_debug_queries() function returns the queries issued during the request as a list. The information provided for each query is shown in Table 16-1.

The after_app_request handler walks the list and logs any queries that lasted longer than a threshold given in the configuration variable FLASKY_SLOW_DB_QUERY_TIME. The logging is issued at the warning level in this application, but in some cases it may make sense to treat slow database alerts as errors.

The get_debug_queries() function is enabled only in debug mode by default. Unfortunately, database performance problems rarely show up during development because much smaller databases are used. For this reason, it is much more useful to enable this option in production. Example 16-2 shows the configuration changes that are necessary to enable database query performance monitoring in production mode.

Example 16-2. config.py: configuration for slow query reporting

SQLALCHEMY_RECORD_QUERIES tells Flask-SQLAlchemy to enable the recording of query statistics. The slow query threshold is set to half a second. Both configuration variables were included in the base Config class, so they will be enabled for all configurations.

Whenever a slow query is detected, an entry will be written to Flask’s application logger. To be able to store these log entries, the logger must be configured. The logging configuration largely depends on the platform that hosts the application. Some examples are shown in Chapter 17.

Tip

If you have cloned the application’s Git repository on GitHub, you can run git checkout 16a to check out this version of the application.

Source Code Profiling

Another possible source of performance problems is high CPU consumption, caused by functions that perform heavy computing. Source code profilers are useful in finding the slowest parts of an application. A profiler watches a running application and records the functions that are called and how long each takes to run. It then produces a detailed report showing the slowest functions.

Note

Profiling is typically done only in a development environment. A source code profiler makes the application run much slower than normal, because it has to observe and take notes on all that is happening in real time. Profiling on a production system is not recommended, unless a lightweight profiler specifically designed to run in a production environment is used.

Flask’s development web server, which comes from Werkzeug, can optionally enable the Python profiler for each request. Example 16-3 adds a new command-line option to the application that starts the web server under the profiler.

This command attaches the ProfilerMiddleware from Werkzeug to the application, through its wsgi_app attribute. WSGI middlewares are invoked each time the web server dispatches a request to the application and can modify the way the request is handled, in this case by capturing profiling data. Note that the application is then started programmatically using the app.run() method.

Tip

If you have cloned the application’s Git repository on GitHub, you can run git checkout 16b to check out this version of the application.

When the application is started with flask profile, the console will show the profiler statistics for each request, which will include the slowest 25 functions. The --length option can be used to change the number of functions shown in the report. If the --profile-dir option is given, the profile data for each request is saved to a file in the given directory. The profiler data files can be used to generate more detailed reports that include a call graph. For more information on the Python profiler, consult the official documentation.

The preparations for deployment are complete. The next chapter will give you an overview of what to expect when deploying your application.

Table of contents collapsed