Download Document

The
Random
Query
Generator
“The Colonoscopy of
Database Software”
Jim Starkey
Philip Stoev
MySQL SystemQA Team
http://forge.mysql.com/wiki/RQG
Testing Coverage [Slutz, 1998]
All Possible SQL Statements and States
Customer Usage Scenarios
Tests
BUGS
What is it?
 The Random Query Generator:
 Generates (pseudo-)random data
 Executes targeted (pseudo-)random queries
 Validates query results and monitors server status
 Reports crashes and other deviations
 http://forge.mysql.com/wiki/RQG
Features
 Multi-platform
- supports Linux, Solaris, Windows (native)
 Fully automatic
- integrated monitoring for unattended runs
- provides pass/fail indication via exit() code and XML
 Highly customizable
- tests all types of queries, tables and fields
- plug-in components for validating the result
 Repeatable runs
- for non-concurrent tests
The SQL Grammar
 Describes the queries to generate,
YACC-style:
 query:

SELECT _field FROM _table where LIMIT _digit
|

UPDATE _table SET _field = _digit where ;
 where:


WHERE _field > _digit |
WHERE _field IS NULL ;
 Provides convenience functions
 _field , _table , _digit, etc.
 Supports Embedded Perl
Query Generation
1. Walk the grammar tree and selectively take one of the
branches at each fork in the road
2. Once a sequence of tokens has been obtained, replace
each with an appropriate value
query:
select:
where:
SELECT
_field
FROM
_table
WHERE
_field
SELECT
`f1`
FROM
`t1`
WHERE
`f2`
> _digit
>
9
Sample Generated Query

SELECT SUM(DISTINCT OUTR . `varchar_nokey` ) AS X
FROM C AS OUTR
WHERE OUTR . `pk` IN (
SELECT INNR . `pk` AS Y
FROM BB AS INNR2
LEFT JOIN BB AS INNR
ON (INNR2.`datetime_nokey` > INNR.`time_nokey`)
WHERE INNR . `int_nokey` > INNR . `int_nokey`
AND OUTR . `time_key` < '2001-04-01‘
)
AND OUTR . `int_nokey` > 1
HAVING X <> '18:18:19‘
ORDER BY OUTR . `varchar_nokey` , OUTR . `pk`;
Testing MySQL @ home
1. Describe the data you wish to operate on
2. Describe the SQL queries that you will be running in a
grammar
3. Run a synthetic workload and monitor server behavior
and performance
Testing storage engines
 Compare query results with a reference storage engine
(MyISAM or InnoDB)
 Ready tests for transactional integrity
- Atomicity, Isolation and Consistency
- REPEATABLE READ
- Durability and recovery
 Stress testing across a wide range of:
- engine configurations
- row counts
- field sizes
- composition of the workload
Testing Recovery

Recovery testing is performed at the end of every test
- recovery is tested “for free” every time
1. Kill the server (with kill -9)
- or kill at important code points via instrumentation
- more sadistic methods forthcoming, e.g. a power cut
2. Restart the server or restore from snapshot/backup
3. Validate the data
- verify that the last transactions were durable
- check that the database remains consistent
- walk the tables back and forth using different methods
- issue CHECK/ANALYZE/OPTIMIZE/REPAIR
Questions?
 Philip Stoev
 Contact: [email protected]
 Code: http://launchpad.net/randgen
 Documentation: http://forge.mysql.com/wiki/RQG