percona-qa

KO | EN

gitlite — search

#javascript #python #hacktoberfest #golang #go #android #rust #linux #nodejs #windows #kubernetes #awesome-list #pytorch #open-source

(★ 70)

Percona QA is a suite of scripts and utilities that assists in building, continuous integration, automated testing & bug reporting for Percona Server, Percona XtraDB Cluster, Percona XtraBackup, Percona Server for MongoDB, as well as other flavors of MySQL (Oracle, Facebook MyQSL, WebScaleSQL, MariaDB) etc.

percona-qa 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

Percona QA
==============================================================================

Percona QA is a suite of scripts and utilities that assists in building,
continuous integration, automated testing & bug reporting for Percona Server, 
Percona XtraDB Cluster, and Percona Server for MongoDB.

For a video introduction to some of the scripts, see:
https://www.percona.com/blog/2015/03/17/free-mysql-qa-and-bash-linux-training-series/

Please Note: For the scripts to run successfully, it is required that sudo is enabled and working and should not request a password.

# Files #

## analyze_crashes.sh ##

This script iterates through MySQL crash core files and runs analyze_crash.sh
on them.

*See Also:* analyze_crash.sh (calls)

## analyze_crash.sh ##

This script takes a MySQL crash core dump as an argument.  It creates a an
output file with a full backtrace of all threads associated with the core and
the time it them backtrace was generated.

*See Also:* analyze_crashes.sh (called from)

## analyze_subdir_cores.sh ##

This script will iterate through all core files starting with the current
working directory and creates report containing full backtrace of all threads
for each in the file gdb_<core file name>_STD/_FULL.txt

## ansible_info.txt ##

Ansible installation/setup information.

## build_mysql.sh ##

Download and build mysql from bzr url.

Example: ./build_mysql.sh lp:mysql-server/5.6

*Note:* only current cmake builds will work,  various compile options can be
set by editing the script.

## build_percona.sh ##

Download and build percona server from bzr url.  Note: script is outdated and
it is preferred to use get_percona.sh (see description)

## build_xtrabackup.sh ##

This script will download and build xtrabackup from a bzr url

Example: ./build_xtrabackup.sh lp:percona-xtrabackup/2.2

## cleanup_ALARMs.sh ##

Deletes ALARM status RQG trials for which there are no errors of any
importance in the server log It does so by checking for which trials ALARM.sh
does not return any important failures (Whether this is the case or not is
determined by the $FINDS and $IGNOR variables in rqg_results.sh)

*See Also:* rqg_failure_counts.sh, rqg_results.sh

## cleanup_failures.sh ##

For use with lp:randgen combinations.pl results

Use with caution. Removes all established known issues for a given
combinations.pl run based on a search string No valid parameters were passed. 
Need a status (1) and a search string to cleanup on (2).  Retry.  Example use:
\$cleanup_failures.sh SERVER_CRASHED 'srv_log_block_size' Note; if you want to
delete all items for a given status, use 'a' (or shortly: a) as the search
string Note; this script relies on the STATUS ./{status passed as first
option} scripts to be present Note; this script relies on the
delete_single_trial.sh script to be present in the same directory as this
script

*See Also:* delete_single_trial.sh (calls)

## codership6-binary.sh ##

Build full and package Percona-XtraDB-Cluster-codership.tar.gz binary
including the galera engine.  Version 5.6

## codership-binary.sh ##

Build full and package Percona-XtraDB-Cluster-codership.tar.gz binary
including the galera engine.  Version 5.5

## COPYING ##

License (GPL 2)

## core_retrieve.sh ##

For a given trial, this script quickly produces a tarball containing: the core
file generated by that trial, the mysqld used by that trial, and the ldd
dependency files needed by that This can be used to quickly get a tarball
which is ready to be added to a bug report for analysis by developers.

Takes the RQG trial number as a parameter.

*See Also:* crash_bug_files.sh (called from), ldd_files.sh (calls)

## crash_bug_files.sh ##

This script quickly gathers all files required for logging a crash bug report.

Takes the RQG trial number as a parameter.

*See Also:* startup.sh (calls), analyze_crash.sh (calls), 
core_retrieve.sh (calls)

## delete_single_trial.sh ##

For use with lp:randgen combinations.pl results

This script deletes a given trial (from a combinations.pl run)
completely.  Execute from the combinations.pl workdir"

Example: to delete trial 1000, execute as:

./delete_single_trial.sh 1000"

*See Also:* cleanup_ALARMs.sh (called from), cleanup_failures.sh (called from),
rqg_results.sh (called from)

## doall.sh ##

This script executes startup.sh in manual RQG mode (without extracting mysql
build and randgen), then runs the resulting cmdtrace script and finally
prepare_reducer.sh.  It expects one parameter: the trial number to analyze

*See Also:* startup.sh (calls), prepare_reducer.sh (calls)

## docker/ ##

Previously in directory ./docker, Percona QA PXC Docker (multi-node Docker 
configurations PXC testing) has been moved to https://github.com/percona/pxc-docker

## docker_info.txt

Docker installation/setup information. Includes information on day-to-day 
tasks, helpful resources etc.

## download_launchpad_single_file.sh ##

This bash script will download a single file from a launchpad project.  See
the usage in the script for more information.   There is a section at the
bottom that can be embedded in another script or Jenkins job if it is known
that the target machine has 'wget' installed.

## download_rpmdeb.sh ##

This script will download all of the rpm/deb packages from the given web
page link.  It is designed to easily download Jenkin's package builds
for testing.  (Requires lynx and wget to be installed)

Example:

download_rpmdeb.sh http://jenkins.percona.com/view/Percona/job/pxc56/999/label_exp=rhel6

## ext/twitter/ip_validator.sh ##

This script will load and test the MySQL stored function check_ip() into a
mysql database.  The command line for running the mysql client is defined in
the environment variable CLI at the top of the script.

check_ip( char(255) ) is an IP address validator.

*The function returns:*

OK (IPV4)       - Good IPV4 address
    OK (IPV6)       - Good IPV6 address
    >!FAIL!(x)<     - Fail where 'x' is a-e designating the type of failure
                          (see function for more info)
    NOT_AN_IP       - Input does not resemble an IPV4 or IPV6 address.

The stored function uses the underlying IS_IPV4(), INET_ATON(), INET_NTOA(),
INET6_ATON() and INET6_NTOA() functions in order to validate the given ip
address.
    
## extract_query.gdb ##

This is a gdb script which is called from prepare_reducer.sh and helps extract
queries that were running from a mysqld core dump.

*See Also:* prepare_reducer.sh (called from)

## get_percona.sh ##

This script downloads recent binaries from the Jenkins CI server (optimized,
with debug symbols and valgrind) and extracts them onto the local system for
testing.  This avoids the time required for local builds and issues with
toolchain variations.  Examine the script to determine or change which
OS/target binaries are being downloaded (i.e.  centos6-64).

## handy_gnu.txt ##

Various shell and command line tips and tricks.

## hung_mysqld.gdb ##

This small gdb command script allows you to analyze the state of a currently
running but hung process.  It will dump all thread backtrace call stacks for
the given process id.

Example:

gdb mysqld {pid} < hung_mysqld.gdb

## keep_single_trial.sh ##

This script keeps a given RQG trial (from a combinations.pl run) completely
(i.e.  all files included) in the KEEP directory within a given
combinations.pl workdir.  Execute this script directly from within the
combinations.pl workdir.  Example: to keep trial 1000, execute as:

./keep_single_trial.sh 1000

## ldd_files.sh ##

This script quickly gathers all shared library dependencies for mysqld and
xtrabackup in the current working directory the script is run from.  The
shared libraries are copied into a *lib64* subdirectory of the current working
directory.

*See Also:* core_retrieve.sh (called from)

## lp-bug-add-file.py ##

This script will add a attached file to an existing launchpad bug report.

This script needs three parameters; the Launchpad bug number, the file to
upload and the file's description.

Usage: ./lp-bug-logger.py -b <bugno> -f <file> -d '<description>'

This script is usually not executed directly, but is instead used by
lp-bug-logger.sh However, if you use it directly, a 'None' output means there
were no errors/that the upload was successful.

You will need to install *python-launchpadlib* for the proper operation of
this script.

*Ubuntu:*

$sudo apt-get install python-launchpadlib

*Others:*

$sudo pip install launchpadlib

Some important information on interaction with Launchpad via the Python
Launchpad API used by this script: The first time you run this script, you
will see something like this (after quite some time):

The authorization page:

(https://launchpad.net/+authorize-token?oauth_token=xxxxxxxxxxxxxx&allow_permission=DESKTOP_INTEGRATION)

should be opening in your browser. Use your browser to authorize this
    program to access LP on your behalf Waiting to hear from Launchpad about
    your decision...

When you see this (and assuming you're working in a text-only ssh connection;
otherwise a browser may have already automatically opened for you), copy and
paste the URL shown into any browser and authorize the app (for example, until
revoked) after logging into LP.  This only has to be done once (if you chose
an indefinite until revoked authorization), and it can be done from any
machine (provided you login to LP).  If you run into any issues, also review
[Unrecoverable error when authorization
fails](https://bugs.launchpad.net/launchpadlib/+bug/814595) if applicable. 
And, authorized applications can be viewed (and revoked) at:
https://launchpad.net/~<your_lp_user_id>/+oauth-tokens

*See Also:* lp-bug-logger.sh (called from)
        
## lp-bug-logger.sh ##

This script uploads, for a given trial, all files in the directory
./BUNDLE_<trialno>/* to a given Launchpad bug report.  It expects two
parameters; the trial number & the bug number.  For Example, to action trial
1000, and upload to bug report 1259034 use:

$./lp-bug-logger.sh 1000 1259034

*Note:* to generate a BUNDLE_<trialno> directory, use crash_bug_files.sh

*Note:* This script calls lp-bug-add-file.py, please review the description of
that script for other important information.

*See Also:* lp-bug-add-file.py (calls)

## mtr_to_sql.sh ##

Scans all MTR testcase commands & generates bughunt RQG grammar/SQL

## multirun_cli.sh ##

Run multiple mysql clients executing the given input SQL multiple times each.

This script expects exactly 5 options. Execute as follows:

$ multirun_cli.sh threads repetitions input.sql cli_binary socket

Where          | Description
:--------------|:------------------------------------------------------------
*threads*      | is the number of simultaneously started cli threads
*repetitions*  | the number of repetitions *per* individual thread
*input.sql*    | is the input SQL file executed by the clients
*cli_binary*   | is the location of the mysql binary
*socket*       | is the location of the socket file for connection to mysqld

*Example:*

$ multirun_cli.sh 10 5 bug.sql /ssd/pxc-5.5/bin/mysql /ssd/test/mysql.sock

*Cautionary Notes:*

- Script does not check *yet* if options passed are all valid/present, make sure you get it right
- If root user uses a password, a script hack is suggested *ftm*
- This script may cause very significant server load if used with many threads
- This script expects write permissions in the current directory (ex. /home/{user}/percona-qa)
- Output files for each thread are written as: multi.<thread number>.<repetition number> (e.g. multi.1.1 etc.)

## multirun_mysqld.sh ##

This script will execute a given saved RQG trial against multiple instances of
a given mysql installation.

This script expects 3 parameters to test server crash with multiple mysqld
processes.

*Parameters:*

1. mysql base directory
2. RQG trial number
3. crashed SQL file name.

*Example:*

$ multirun_mysqld.sh /ssd/ramesh/pxc-5.6.20-x64 36 in.sql

*See Also:$ myextra.sh (calls)

## myextra.sh ##

This script outputs the extra mysqld options used in a given RQG trial. It
expects one parameter: the trial number to analyze.  Ensure that startup.sh
has already been executed for this trial.

*See Also:* multirun_mysqld.sh (called from), prepare_reducer.sh (called from)

## packaging_testing/ ##

The scripts in this directory will test packaging across supported OS distributions.

## parse_general_log.pl ##

This script can be used to extract SQL from a MySQL general-log.

*Usage:*

[perl] ./parse_general_log.pl -i infile -o outfile

*Where:*

- infile is a standard mysqld general query log
- outfile is the SQL output extracted from the log.

General mysqld logs can be generated with these options to MTR (or RQG):

--mysqld=--log-output=FILE --mysqld=--general_log --mysqld=--general_log_file=out.sql

*WARNING:* by default this script eliminates a number of statements which may
cause replay issues (KILL, RELEASE) This may lead to non-reproducibility for
certain bugs.  Check the script's source for specifics

### percona_repo_install.sh ###

This script will enable the Percona apt/yum repository on a RHEL,Centos,
Debian or Ubuntu server.  The script must be run with root permissions.

If the script is hosted at a public URL then it can install
the Percona apt/yum repo with a single command:

# curl -s {url}/percona_repo_install.sh | bash /dev/stdin

## performance.sh ##

*Note:* Jenkins CI script

This is a performance testing script that is designed to be run from the
Jenkins Continuous Integration system.  It will run a series of tests with the
MySQL Test Suite and Sysbench both with default and tuned settings.

It requires two parameters.

1. relative workdir
2. relative basedir

Usage example:

$./performance.sh 100 Percona-Server-5.5.28-rel29.3-435.Linux.x86_64

This would lead to /data/bench/qa/100 being created, in which testing takes
/place and data/bench/qa/Percona-Server-5.5.28-rel29.3-435.Linux.x86_64 would
/be used to test.  The fixed root workdir to which all options are relative is
/set to /data/bench/qa in the script.

## performance-old.sh ##

*Note:* Jenkins CI script

This is an older version of the performance.sh script.  It takes the same
parameters.

*See:* performance.pl

## pquery/ ##

A multi-threaded stress test for mysqld.  This utility will read SQL queries
from a file (one per line) and spawn a given number of threads that will
execute the queries from the list in random order.  The number of queries each
thread will execute is equal to the SQL input line count but it is not
guaranteed that all queries will run as they are selected at random, some
queries may be run multiple times and others not at all.

Also included is a generator script which maybe used to generate queries for
pquery.  Other tools such as randgen can also be used to generate random query
data for pquery input.

*See Also:* pquery-sb/

## pquery-sb/ ##

pquery-sb is a lua script to be used with the sysbench application.  It
performs the same function as the C++ based pquery utility.

*See Also:* pquery/

## prepare_reducer.sh ##

A wrapper for /util/reducer/reducer.sh in the lp:randgen source tree.

This script creates a local run script for reducer. It expects one parameter:
the trial number to analyze.

To aid with correct bug to testcase generation, this script creates a local
run script for reducer and sets #VARMOD#.  This handles crashes/asserts for
the moment only.  Could be expanded later for other cases, and to handle more
unforeseen situations.

## process_signals.sh ##

script which displays process info for pid along with information on which
signals are pending, blocked, ignored and caught by the process.

## ps-async-repl-test.sh ##

*Note:* Jenkins CI script

This script will help us to test different asynchronous replication methods like master slave, master multi slave, master master, multi source replication, multi thread replication etc

## pxc56-mtr.sh ##

This script is designed to run from the Percona Jenkins job
named 'pxc56-mtr'.   It runs selected MTR tests on a recent
PXC 5.6 build.

## pxc55to56.sh ##

*Note:* Jenkins CI script

2 node cluster testing script, also tests upgrade from 5.5 to 5.6

This script tests a two node PXC 5.5 cluster, runs the mysql test suite and
sysbench against it, then upgrades the cluster to 5.6 and re-runs the MySQL
test suite with and without compatibility options, runs sysbench under 5.6 and
then outputs the versions and row count of the test table on both nodes.
                            
## pxc56to56.sh ##

*Note:* Jenkins CI script

Tests cluster combining 5.6 release with 5.6 latest.

Tests cluster SST and IST using Mysql test suite and multi-table sysbench from
both nodes, performs an mysql_upgrade retests.  Reports versions and row
counts at the end.

## pxc56to56-t2.sh  (take 2?) ##

Alternative version of pxc56to56.sh testing from one release version to
another automatically downloads and builds sysbench if not found, wsrep
debugging disabled, common parameter ordering changed.

## pxc6-bench.sh ##

*Note:* Jenkins CI script

This is a benchmark script for Percona XtraDB Cluster 5.6.  It benchmarks
using MySQL Test Suite, sysbench and has support for dual node cluster
testing.

## pxc6-rqg.sh ##

*Note:* Jenkins CI script

This is an automated randgen testing script for PXC 5.6.  Resulting randgen
trials are stored for later analysis.

## pxc6-sst-test.sh ##

*Note:* Jenkins CI script

This script tests Galera snapshot transfer between nodes in a cluster
configuration.  For PXC 5.6.

## pxc-bench.sh ##

*Note:* Jenkins CI script

Earlier version of the PXC benchmarking script.

*See:* pxc6-bench.sh

## pxc-failure-test.sh ##

This script is for PXC Testing ChaosMonkey Style Please go through [the
tutorial](https://github.com/jayjanssen/percona-xtradb-cluster-tutorial) to
install PXC on your machine and execute this script from node1 Analyze
script_out.log file from workdir to get the PXC testcase output

## pxc-replication-check.pl ##

This script is designed to monitor IST replication state between PXC wsrep
nodes for various SQL instructions.  An example of a bug this will test is for
is lp1421360.  For a reliable test please make sure there is no other traffic
on the test cluster.

## pxc-prepared-stmnt-test.sh.sh ##

This script is made for PXC Prepared SQL Statement test

## pxc-rqg.sh ##

*Note:* Jenkins CI script

Earlier version of the pxc6-rqg.sh randgen script.

*See:* pxc6-rqg.sh

## pxc-sst-test.sh ##

*Note:* Jenkins CI script

Earlier version of the pxc6-sst=test.sh

*See:* pxc6-sst-test.sh

## reducer.sh ##

reducer.sh - the best mysql testcase reduction program

For more info, search Google for:   reducer.sh

## rqg_failure_counts.sh ##

*Note:* Jenkins CI script

Jenkins CI script to check randgen failure counts.

## rqg_results.sh ##

This script handles various trial outcomes for a given RQG run (run from the
RQG combinations.pl workdir containing all trialx.log files)

*Note:* this script relies on the delete_single_trial.sh script to be present
in the same directory as this script

If you start rqg_results.sh with WIPE as the first option ($./rqg_results.sh
WIPE) it will wipe all STATUS_ENVIRONMENT_FAILURE, all STATUS_INTERNAL_ERROR,
and all STATUS_PERL_FAILURE failed trials by default

## run_times_parse.sh ##

parse_times.sh allows one to check the start and end times for all trials
within a RQG combinations.pl run direct This allows for quick analysis to see
if any of the trials stalled/hanged/locked up by reviewing the durations

## setup_devtoolset.sh ##

*Deprecated*

This script does not work, leaving here for future work or future deletion

## setup_server.sh ##

This script makes major changes to any machine it is executed on (installs
packages, enables core files, changes significant security settings etc.). 
Please review the planned changes by reviewing the script contents.  All of
the changes in the script prepare this machine to be a proper QA server.  This
script is meant for use on Centos (6.x mainly).  Make sure that you want to
continue.

*Note:* this script removes pulseaudio drivers.  If you are using a
desktop/would like to keep pulseaudio, please remark the line in the script
first.  Finally, this script can be executed multiple times without doubling
up configurations/without messing up anything.  It was specifically coded for
this.  This script requires sudo and su commands to be available already.  Use
visudo as described above (in 'Sudo:') to add yourself to sudo list (faster
run).  This script is provided AS-IS with no warranty of any kind.  Do NOT run
on production machines.  This script lowers the servers security.  Use at own
risk.

## sequence_match.pl ##

This script will search through the standard input for a sequence of lines
that match a sequence of regular expressions and report when a match is found.

This can be used to find blocks of matching code within a project.

## startup.sh ##

Generate cmd startup scripts used in an RQG trial.

This script expects the trial number to be passed - e.g. 'startup 10' for
actioning trial 10 (trial10.log) Other uses: 'startup 10 man': actions trial
10 as found in the RQG run without attempting to extract any tars Other uses:
'startup 0': local build mode (create scripts for a standard server setup
without reference to RQG)

In other words, use:

$ startup.sh <trialno>         When analyzing automated Percona Jenkins RQG runs
    $ startup.sh <trialno> man     When analyzing manually executed RQG runs
    $ startup.sh <trialno> man2    Like man option above, but do not attempt to untar tarball again
    $ startup.sh 0                 When having extracted a binary tar build with ./build/build-binary.sh (no RQG)

## test_mysqld_options.sh ##

This script quickly tests all mysqld options. Note it expects cores to be
written to /cores/core.pid.  Location can be changed in-script, but not
filename To set your server up in this way (in terms of corefile generation),
see core file setting part of setup_server.sh, available at lp:percona-qa

percona-qa

percona-qa 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

percona-qa

percona-qa 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README