How SQLite Is Tested

Summary

How SQLite Is Tested Table Of Contents 1. Introduction The reliability and robustness of SQLite is achieved in part by thorough and careful testing. As of version 3.42.0 (2023-05-16), the SQLite library consists of approximately 155.8 KSLOC of C code. (KSLOC means thousands of "Source Lines Of Code" or, in other words, lines of code excluding blank lines and comments.) By comparison, the project has 590 times as much test code and test scripts - 92053.1 KSLOC. 1.1. Executive Summary Four independently developed test harnesses 100% branch test coverage in an as-deployed configuration Millions and millions of test cases Out-of-memory tests I/O error tests Crash and power loss tests Fuzz tests Boundary value tests Disabled optimization tests Regression tests Malformed database tests Extensive use of assert() and run-time checks Valgrind analysis Undefined behavior checks Checklists 2. Test Harnesses There are four independent test harnesses used for testing the core SQLite library. Each test harness is designed, maintained, and managed separately from the others. The TCL Tests are the original tests for SQLite. They are contained in the same source tree as the SQLite core and like the SQLite core are in the public domain. The TCL tests are the primary tests used during development. The TCL tests are written using the TCL scripting language. The TCL test harness itself consists of 27.2 KSLOC of C code used to create the TCL interface. The test scripts are contained in 1390 files totaling 23.2MB in size. There are 51445 distinct test cases, but many of the test cases are parameterized and run multiple times (with different parameters) so that on a full test run millions of separate tests are performed. The TH3 test harness is a set of proprietary tests, written in C that provide 100% branch test coverage (and 100% MC/DC test coverage) to the core SQLite library. The TH3 tests are designed to run on embedded and specialized platforms that would not easily support TCL or o...