dbdeployer downloads add-remoteno longer limits MySQL, MySQL Cluster and MySQL Shell to a fixed list of release series (issue #149): any series in the formmajor.minoris accepted (e.g.9.7,26.7), and the download site decides whether it exists. New releases no longer need a dbdeployer update.add-remotefor a series that the download site does not list (for example a retired innovation release such as 9.2) used to register the tarballs of the current default release instead, because the site answers such requests with the default page. Only tarballs that belong to the requested series are now registered; otherwise the command fails and names the series the site offered.- MySQL 26.x (calendar versions, starting with 26.7) is now treated as MySQL.
Flavor-blind version checks treated every major from 10 as MariaDB, so
26.x got pre-8.0 behaviour:
downloads get-by-version 26.7failed, and the generatedtest_replicationchecked the old replication status columns. The replication syntax (SHOW REPLICA STATUS, CHANGE REPLICATION SOURCE, SHOW BINARY LOG STATUS, RESET BINARY LOGS AND GTIDS, semisync source plugins, 8.4 group replication options) is now decided by flavor capabilities: the MySQL family (MySQL, Percona Server, PXC, NDB, VillageSQL) gets it from the respective version, MariaDB never does.common.GreaterOrEqualVersionis no longer used by dbdeployer. - Group replication sandboxes set
group_replication_communication_stack=XCOM(8.4+). Each node uses a dedicated group communication port, which the MYSQL stack, the default from MySQL 26.x, does not support: nodes stayed OFFLINE. add-remoteregisters ARM and MySQL Shell tarballs with the architecture names used by the registry (arm64,amd64), so they are found byget-by-versionon those platforms (they wereaarch64,x86-64,arm-64).- The
startscript of a ProxySQL sandbox now waits until the admin interface accepts connections (up to 30 seconds), instead of returning 2 seconds after the PID file appears. On a slow host the admin port could still be closed whendeploy proxysqlreturned, so the firstusefailed withERROR 2003 ... (111).
- Tarball registry: MySQL 9.7.2 and 26.7.0 (Linux x86_64 and aarch64, macOS).
- CI deploys MySQL 9.7.2 and 26.7.0 (single, replication, group replication, fan-in, all-masters, get-by-version), and 26.7.0 with InnoDB Cluster and ProxySQL.
- Fix a ~2 minute wait per node when deploying MariaDB without Galera (issue #142): a 3-node MariaDB replication deploy took about 8 minutes. The wsrep readiness wait is now generated only for Galera and PXC nodes, instead of guessing at runtime whether the server is a cluster node; other sandboxes no longer query the server for it at all.
- A Galera or PXC node that does not become ready (
wsrep_readyON) within 120 seconds now fails the deploy with an error naming the node and its log, instead of continuing and failing later at grants. - Joining Galera and PXC nodes are now actually waited on. They receive the
first node's grants by SST, so the wait's passwordless root login was
rejected; after 30 seconds the wait gave up silently and the deploy went
on without checking
wsrep_ready. Joining nodes now log in with the sandbox credentials.
- Deploy MariaDB 11.4.13, 12.3.3, and 13.0.2 (in addition to 10.11.9) in the
MariaDB integration job, asserting
CHANGE MASTER TOreplication syntax and a deploy time budget for each (issues #139, #140, #141). test/deploy-time-budget.shacceptsDEPLOY_VERSIONto test a specific version, and always runs theDBDEPLOYER_BINARYit was given.
- List the MariaDB versions verified in CI (10.11.9, 11.4.13, 12.3.3,
13.0.2) with their minimum glibc, and document fetching MariaDB releases
that are not in the registry with
dbdeployer downloads get-by-version X --flavor=mariadb --unpack.
- Fix excessive wait times after
deploy replication: deployments that took about 5 minutes in 2.4.0 now complete in seconds (issue #131). The wsrep readiness check no longer spins on non-Galera servers, and the replica-readiness poll is bounded and authenticates as the sandbox user. deploy postgresqlnow honors--sandbox-home,--port,--sandbox-directory,--db-user, and--db-password, enabling per-project and multi-sandbox PostgreSQL deployments. Behavior is unchanged when no flags are provided.- Use a flavor-appropriate default prompt: MariaDB sandboxes now show a
MariaDBprompt instead of alwaysmysql.
- Add a time-bounded
deploy replicationregression test (deploy-time-budget) to catch wait-time regressions.
- Add the PostgreSQL
test_replicationscript and replication data-flow verification. - Add MariaDB Galera 11.8.6 to the tested configurations and CI coverage.
- Wait for PostgreSQL replicas to be ready to serve queries before returning from deployment (issue #131).
- Prevent PostgreSQL port collisions across single, replication, and multiple deployments.
- Fix PostgreSQL + ProxySQL ports, scripts, and sandbox-directory handling.
- Fix PostgreSQL deb extraction layout and sharedir relocation, including the
required
libpq5runtime dependency. - Treat MariaDB 11+ versions correctly in version comparisons and replication authentication.
- Retry sandbox deletion when a killed process still holds files open.
- Improve install-script dependency diagnostics and checksum handling.
- Stabilize PXC and MariaDB Galera startup and macOS test execution.
- Require Go 1.23 in CI and add regression coverage for replica readiness.
- Fix the install-script test's release lookup when running without a checkout.
- Add MariaDB downloads through the MariaDB Foundation API, including Galera support and MariaDB 11.8 coverage.
- Add PXC and MariaDB Galera integration tests and tested configurations.
- Add MariaDB flavor template coverage and derive the application version from git tags.
- Improve MariaDB client handling, minimal-download fallback, and checksum detection.
- Stabilize Galera and PXC CI dependency setup and replication templates.
- Add CI badges and the tested configurations table.
- Add Galera and PXC provider documentation.
- Fix arbitrary file write via chain-symlink path traversal during tar
extraction. A malicious archive could bypass the earlier symlink
guard added for CVE-2020-26277 by chaining relative
dirN -> dirN-1/..symlinks whose cumulative realpath escaped the extraction directory; a later regular-file entry could then be written through the escaping symlink. Each entry's path and each symlink's would-be target are now resolved through the filesystem withfilepath.EvalSymlinksand rejected if they leave the extraction directory; dbdeployer also refuses to overwrite an existing symlink with a regular file. Only malicious archives are affected: officially distributed MySQL / MariaDB / Percona / etc. tarballs do not contain such symlink chains. Reported responsibly by an external security researcher (PR #92).
- Fail early when
mysqlshon PATH is too old for the target server version (e.g. mysqlsh 8.0.x + MySQL Server 8.4+), instead of letting the AdminAPI fail mid-deploy with "Unsupported server version" (issue #87). dbdeployer now parsesmysqlsh --version, compares its major.minor to the server's, and emits an actionable error telling the user to install a matching mysqlsh.
- Recommend
sudo cpinstead ofsudo mvin manual install steps, so thedbdeployerbinary in/usr/local/bin/is owned by root rather than the user who ranwget/tar(issue #89).
- Add VillageSQL flavor support. VillageSQL is a MySQL drop-in replacement
with extensions. Its tarballs are detected via the unique marker file
share/villagesql_schema.sqland reuse MySQL's sandbox lifecycle (init, start, stop, grants, replication) unchanged. Because VillageSQL uses its own version scheme (e.g. 0.0.3), unpacking requires--unpack-versionmapped to the MySQL base version (e.g.--unpack-version=8.0.40). - Add VillageSQL to CI: real tarball download, SHA256 checksum verification, single and replication deployment tests, capability inheritance tests.
- Fix InnoDB Cluster deployment: let mysqlsh manage Group Replication from scratch instead of conflicting with dbdeployer's GR setup
- Fix InnoDB Cluster Basedir template pointing to wrong directory
- Fix
--with-proxysqlfailing for InnoDB Cluster (wrong sandbox path in ProxySQL config) - Fix Router port extraction including config file path in the result
- Fix Router start hanging forever when mysqlrouter process forks
- Fix ProxySQL grep -v exiting under
set -ein monitoring scripts - Fix ProxySQL GR monitor seeing all nodes as offline (hostgroup 3)
- Fix fan-in CREATE DATABASE on node2 conflicting with node1's database
- Fix copy of mysqlsh lib/mysqlsh/ directory (only .so files were copied)
- Fix PostgreSQL multiple sandbox directory naming
- Fix symlink opt/mysql for ts tests in CI (HOME path mismatch)
- Remove MariaDB 11.4 from CI (authentication bug #82)
- Add install script test workflow (downloads and verifies dbdeployer install)
- Drop macos-13 from install test (unsupported runner)
- Replace sleep+check with retry loops for replication verification in CI
- Add MariaDB 10.6/10.11/11.4/11.7/11.8 and Percona Server 5.7/8.0/8.4 to tarball registry
- Add ts replication test suite (MySQL 5.7, 8.0, 8.4, 9.5)
- Add PostgreSQL ts testscript tests (single + replication)
- Add MySQL 9.5 support for semisync and ts replication tests
- Use Slave|Replica pattern for IO/SQL thread status in multi-source ts tests (compatibility with MySQL 8.x terminology)
- Resolve CI failures for MariaDB, Percona, ts replication, and fan-in
- Add Percona Server and MariaDB integration tests
- Add PostgreSQL ts testscript tests to CI
- Add ts replication test suite to CI (5.7, 8.0, 8.4, 9.5)
- Fix macOS --minimal fallback and
--guessusing real URL patterns - Fix install script to download checksums.txt instead of per-file .sha256
- Add InnoDB Cluster topology (
--topology=innodb-cluster) with MySQL Shell and MySQL Router support - Add ProxySQL GR-aware hostgroups for InnoDB Cluster and Group Replication
(
--with-proxysqlconfigures reader/writer hostgroups automatically) - Add
--topology=groupsingle-primary and multi-primary Group Replication with full CI coverage - Add fan-in and all-masters replication topologies with data verification
- Add
downloads add-urlcommand for custom tarball URLs - Add MySQL 8.4-specific replication and group replication templates
- Add ProxySQL PostgreSQL backend wiring (
pgsql_servers/pgsql_users) - Add
--providerflag and PostgreSQL routing to all deploy commands - Add
dbdeployer deploy postgresqlstandalone command - Add
dbdeployer init --provider=postgresqlfor one-command setup - Add macOS PostgreSQL support via Postgres.app binary detection
- Add cross-database topology constraint validation
- Add comprehensive topology/provider/proxy reference documentation
- Add group replication, fan-in, all-masters, PostgreSQL multiple tests to CI
- Add InnoDB Cluster integration tests (MySQL 8.4.8 + 9.5.0)
- Add functional verification (write/read) to all integration tests
- Add ProxySQL
--bootstrapmode test script - Add admin web UI proof of concept
- Replace
\Gwith--verticalin all replication templates (MySQL 9.5 compat) - Fix semisync template variable scoping and version detection
- Fix PostgreSQL deb extraction version detection and binary setup
- Fix PostgreSQL initdb requiring empty data dir (create log dir after initdb)
- Fix PostgreSQL share files for deb-extracted binaries (timezonesets path)
- Remove dead commented-out
semisync_master_enabledfrom template - Fix
gosecandstaticchecklint warnings in PostgreSQL and ProxySQL code
Initial release under the ProxySQL organization. Forked from datacharmer/dbdeployer v1.73.0 with Giuseppe Maxia's blessing.
- PostgreSQL provider: full provider architecture with
initdb, config generation (postgresql.conf,pg_hba.conf), single sandbox deployment, streaming replication viapg_basebackup, and monitoring scripts - PostgreSQL deb extraction for binary management (
unpack --provider=postgresql) - ProxySQL provider: standalone and topology-integrated deployment
(
--with-proxysqlwires read/write split into any MySQL/PostgreSQL topology) - Provider interface with
SupportedTopologiesandCreateReplica - Add MySQL 8.4.0–8.4.8, 9.0.1, 9.1.0, 9.2.0, 9.3.0–9.5.0 to tarball registry
- Add
dbdeployer initwith curl-based install script - Add admin web UI proof of concept
- Add comprehensive website documentation at proxysql.github.io/dbdeployer
- Full GitHub Actions CI pipeline: lint, unit tests, build verification
- Integration tests: MySQL, Percona Server, MariaDB, PostgreSQL, InnoDB Cluster, Group Replication, fan-in, all-masters, ProxySQL wiring
- Install script test workflow across multiple OS versions
- Add tarballs for MySQL 8.0.33 to downloads list
- Fix security vulnerabilities in dependency (golang.org/x/text and golang.org/x/term)
- Fix security vulnerability in dependency (golang.org/x/text)
- Update user-agents used in
downloads add-remoteto fix failed recognition by MySQL.com downloads servers. The new user agent identifies the request as coming fromdbdeployer, instead of pretending to be a browser.
- Add option
--change-user-agentfor commanddownloads add-remoteto use alternative user agent in case of failure. If used, this option changes the user agent so that the request comes from a fake Firefox browser with a very high version number (i.e. the current year). - Add tarballs for MySQL 8.0.32 to downloads list
- Update code of
show_binlogscript to find the right binary log with version 8.0.x, when used without--masteroutside replication.
- Add downloads for MySQL 8.0.31 and Percona Server 8.0.27, 8.0.28, 8.0.29
- Add option
--merge-importedtodbdeployer downloads import
- Fix
goseccomplaints in tests
- Fix Issue #156 : unquoted directory names in shell scripts cause failure
- Fix error in data-load: getting 'menagerie' database into a replicated sandbox
requires all nodes to set
local_infile=ON
- Fix error in cleanup procedure for composite sandbox (The cleanup was not able to delete a non-empty directory)
- Add testing command
check_sandbox_manifest. - Improve TestMain with a check that prevents double buildUp or early tearDown
- Add downloads for arm64 (Issue #145)
- Add option
--archtodownloadscommands:get-by-version,add(mandatory),list,tree - Released binaries for MacOS are now universal (
ARM64andX86_64)
- Add option
--retries-on-failureto downloadsget*andimportcommands - Add file
connection_super_user.confandconnection_super_user.jsonto each sandbox
- Reformat code using go 1.19 (some tiny differences are found within comments)
- Remove deprecated usage of
io/ioutil(replaced with eitherosorio) - Add testscript tests (
tsandts_static)
- Remove from downloads list all 8.0.29 tarballs, after they were removed from MySQL downloads archives.
- Add unpack capability to
downloads getanddownloads get-by-version. These commands can use all the options allowed toget-unpack. - Add downloads for MySQL 8.0.29 minimal (was skipped in 1.65.0).
- Add downloads for MySQL 8.0.30.
Isolate functions for several commands
These changes move the main functionality from the cmd package
(using cobra.Command and args) to the ops package (using a structure)
This change makes it easier to test, and to use dbdeployer as a library from other
packages.
- Fix issue #148: incorrect default value listed for option
--OS - Fix isste #149: port calculation exceeds 64K
- Fix issue #147:
sysbench_ready rundoes not use env variable
- Add downloads for MySQL 8.0.28 and 8.0.29 (PR #146 bt @isotopp)
- Fix JSON label missing quote in
data_load.go
- Improve comments and tests for
regexp_util - Skip
gosecurewarnings in in ternally used functions. - Improve testing code using
t.Fatalandt.Skip
- Add downloads for MySQL 8.0.27
- Add downloads for MySQL 8.0.23 (Issue #144)
- Removed CRC from sample databases provided by MySQL.com: the CRC are not reliable, as they change often.
- Fixed staticcheck warnings
- Fix Issue #140 - Wrong command suggestion
- Fix Issue #137: missing check on .dbdeployer dir when no sandboxes were created
- Fix Issue #138 : Update data load URLs and CRCs
- Update build directives to work with go 1.17
- Add downloads for MySQL 8.0.26
- Add command
downloads tree(shows tarballs by version) - Add option
--sort-bytodownloads list
- Deprecated support for TiDB database. There is no interest in this flavor, and the TiDB community has developed its own tool to achieve the same goal.
- Command
downloads getcan also use direct URL instead of tarball names - Add (hidden) command
downloads add-stdin - Add command
downloads add-remote - Add option
--versiontodownloads list
- (@lefred) Add MySQL 8.0.24 binaries to downloads
- (@lefred) Add
./scripts/get_dbdeployer_mysql_json.sh
- (@ogrovlen) Allow other custom mysqld than mysqld-debug
- Replace internal template variables with external files (required Go 1.16)
- Remove
_templatesuffix from most template names
- Fix deletion of specified sandbox when there is only one. Sandbox name was not checked
- Prevent arbitrary symlinks from being extracted with tarballs
- Fix Issue #124 (Can't reset defaults after an upgrade)
- Add script
wipe_and_restartto single sandboxes - Add scripts
exec_all,exec_all_masters,exec_all_slaves,wipe_and_restart_allto replication sandboxes - Add subcommand
execto commandglobal.
- Add subcommands
import,export, andresetto commanddata-load
- Add options
--lsand--runto commanduse - Add scripts
sysbenchandsysbench_readyto all sandboxes - Add command
data-loadwith subcommandslist,show,get
- Fix Issue #120 (Can't SELECT the mysql.user table in mariadb 10.4 or later)
- Add second optional argument to
dbdeployer useto indicate the executable to run within the sandbox.
- Fix Issue #118 (dbdeployer fails to autodetect pxc flavor from binaries of 5.6.x versions)
- The
send_killscript accepts an argumentdestroy(aliases-9orcrash) to halt the server immediately. - The
deletecommand is much faster (usessend_kill destroy)
- The
usecommand should re-start a sandbox that was stopped, but it was not doing it (Issue #116) - Listing of small tarball for 8.0.21 lacked the 'minimal' attribute
- Added MySQL shell 8.0.21 to downloads list
- Improve
all_tests: Now it doesn't require the version - Improve unit test script to detect where to run
- Fixed issue #115: failing to detect missing
socatfor PXC
- Fixed bug during
init. When the sandbox-binary directory existed, but empty, the download was skipped. dbdeployer downloads exportwould not export the fielddate_added. Such field was lost during build, due to a missed field incode-generation.go
- Added MySQL 5.7.31 to downloads list
- Added test for
dbdeployer initunder Docker containers for Ubuntu 18, Ubuntu 20, CentOS 7, CentOS 8
- Fix bash completion for CentOS (missing completion script and unchecked
sudocall)
- Add command 'use' and sandboxes options
- Add options
--by-date,--by-flavor,--by-version,--newest,--oldestto commandsandboxes
- Add latest MySQL tarballs to downloads list
- Add Percona Server minimal to downloads list
- Fix issue #111 - allows using NDB 7.4/7.5
- Add command
initthat initializes dbdeployer environment- creates
$SANDBOX_HOMEand$SANDBOX_BINARYdirectories - downloads and expands the latest MySQL tarball
- installs shell completion file
- creates
- Fix error handling on default directories detection for bash completion.
- Fix bug in defaults handling: "default-sandbox-executable" was not used in
info defaults - Prevent
lddfrom running on mocked binaries
- Moved mock tests under docker to improve consistency
- Add default sandbox (commands
admin set-defaultandadmin remove-default)
- Fix incorrect privileges when updating dbdeployer as root
- Fix Issue #108: panic error when sandbox binary is not a directory
- Add new options
--rawand--statsto commandinfo releases - Add checksum files (SHA256) to release
- Add signature check during dbdeployer update
- Add
--change-master-options(Issue #107) - Add more downloads (8.0.20, 5.7.30)
- Add more tests
- Docker test now downloads latest 8.0 binaries instead of using built-in.
- Improve PXC handling of defaults across versions
- Fix defaults for PXC 5.6, which were broken after introducing support for PXC8.0 (Issue #106)
- Add
pxc_encrypt_cluster_trafficoption to PXC options file to adjust to recent default changes and allow PXC 8.0.18 to work out of the box
- Change all explicit
bashpaths in shell scripts toenv bash
- Add ability to deploy PXC 8.0 without using any options (dbdeployer
uses
wsrep_sst_method=rsyncfor 5.7 andwsrep_sst_method=xtrabackup-v2for 8.0)
- Add option
--task-userand--task-user-roleto create additional user with custom role. - Add cookbook recipe
single-custom-users(include sample creation of orchestator user)
- Add security checks with
gosecscans - Update copyright notice
- Add test for cookbook scripts
- Add custom role to 8.0 sandboxes
- Add role related options:
--default-role,--custom-role-name,--custom-role-privileges,--custom-role-target,--custom-role-extra - Add filters to
globalcommand:--type,--flavor,--version,--short-version,--name,--port,--port-range. Each option can also be negated (--type='!single',--flavor=no-mysql,--port='!8019') - Add verbose and dry-run flags to
global. - Add
metadata_all_templatescript to composite sandboxes - Add command
downloads add - Add tarballs for 5.7.29, 8.0.19 for MySQL and NDB (Linux and MacOS)
- Add test for tarball integrity and reachability
- Fix bug in NDB cluster: the cluster name was not reported correctly in the NDB configuration file
- Fix determination of shell interpreter
- Add 'IF NOT EXISTS' to 'CREATE ROLE', to account to changed behavior of NDB server in 8.0.18
- Fix for hanging during library check (Issue #99)
- Explicit template for PXC replication options (Issues #87 and #92) Allows installing PXC with version 5.6
- Explicit template for Group replication options
- Fix command "dbdeployer info version --flavor VER" It would not display the requested short version but the latest one.
- dbdeployer can guess the next downloadable version with the command
dbdeployer downloads get-by-version 8.0 --guess-latestordbdeployer downloads get-by-version 5.7 --guess-latest
- Issue #95 mismatched values in fan-in when using nodes > 3
- Issue #93 Client detection for imported sandbox doesn't take flavor into account
- Add test for
FindOrGuessTarballByVersionFlavorOS - Add mock test for fan-in nodes
- dbdeployer now detects when essential libraries are missing (Linux only)
- New flag '--skip-library-check' will avoid the above automated detection but will expose to the risk of having an incomplete sandbox if the prerequisites were not satisfied.
- Add tarballs for 8.0.18 and 5.7.28 to the downloads list
- Add check to verify tarball correctness (they must contain a directory that matches the tarball name)
- Show stderr text when external commands fail - Issue #94
- Enable start.log by default.
- Add missing exit directive on failure in
init_dbtemplate - Improve error messages on
init_dbfailure - Build Linux amd64 instead of 386
- Add script
clone_fromto clone from another 8.0.17+ server - Add option "clone" to script
replicate_from - Add ability of detecting and using GTID to script
replicate_from - Add ability of finding a suitable client for imported sandboxes
- Add option --server-id to "deploy single" command
- Add option --base-server-id to deploy command
- Add option --sandbox-directory to import command
- add command dbdeployer import single
- Enable Go modules
- Remove vendor directory
- Change calls to StringSliceP with StringArrayP
- (No changes in behavior. Some flags help items will display "stringArray" instead of "strings")
- Fix a bug in 'dbdeployer update': it only worked with --verbose enabled
- Fix output of 'dbdeployer update': it now confirms that the update has happened
- Add command "dbdeployer update"
- Add command "dbdeployer info releases"
- Fix 'dbdeployer info defaults': some values were not reported
- Add test for defaults info
- Improve command "dbdeployer downloads import": it can also import from a URL
- Add downloads binaries for 8.0.17 and 5.7.27
- Add mysql-shell binaries in downloads
- More tests for
common/*
- Add command "defaults enable-bash-completion" to help install bash completion
- Add command "defaults flag-aliases" to show aliases for some flags
- Add flag "--earliest" to command "info version"
- Downloads list should show files for current OS only
- Fix tarball list generation to be rebuilt only when contents change
- Fix OS user detection to rely on operating system library calls instead of environment variables.
- Replace
sort_versionswith calls to "dbdeployer info version"
- Add command "downloads get-unpack" to combine the effects of "dbdeployer downloads get" and "dbdeployer unpack" into a single command (Issue #81)
- Issue #83 'unpack --overwrite' should not ask for confirmation
- Issue #84 '--sandbox-binary' does not expand '~' to $HOME
- Add option
--shell-pathto choose alternative Bash interpreter used in all scripts, including cookbook recipes and mock scripts
- Add test for alternative Bash interpreter
- Add command 'downloads'
- add subcommand 'downloads list'
- add subcommand 'downloads get'
- add subcommand 'downloads get-by-version'
- add subcommand 'downloads import'
- add subcommand 'downloads export'
- add subcommand 'downloads reset'
The command 'remote' is deprecated with its subcommands Its functionality is replaced by 'downloads', albeit with a different syntax.
- Fix issue #80 NDB tarballs are wrongly labeled as regular "mysql" flavor
- Fix Issue #72 Catalog operations can clash with multiple dbdeployer runs
- Fix wrong description for "defaults update"
- Fix wrong description for "cookbook create"
- Fix option
--sort-byincorrectly assigned to command "cookbook" instead of subcommand "cookbook list"
- Add command "dbdeployer info" to improve dbdeployer usage in scripts
- Add test for parallel usage
- Reinstate parallel deployment in functional test
- Improve replication between sandboxes test, by checking for existing versions better.
- Change upgrade procedure for MySQL 8.0.16+
- Adjust cookbook upgrade recipe to use verbose output
- Add options
--dry-runand --verbose to upgrade command
- Enhance export tests
- Add missing error checks
- Clean code that was failing static checks
- Add export tests
- Add conditional static check to sanity test
Add missing $ in recipe skip-start-replication
- Add option --sort-by=[name,flavor,script] to cookbook list
- Add option --flavor for versions command
- Add command export to show dbdeployer command structure as JSON
- Add cookbook recipes
- Single sandbox deployed without starting the server
- Replication sandbox deployed without starting the servers
- Replication sandbox with custom names for directories and scripts
- Single sandbox with admin address enabled
- Fixed plural of custom replication names
- Add cookbook recipes:
- replication across multiple versions
- replication from group replication to single sandbox
- replication from group replication to master/slave
- replication from single sandbox to group replication
- replication from master/slave to group replication
- replication between single sandboxes
- circular replication
- Add custom
replication_fromscript to each sandbox. - Add metadata script to each sandbox
- Add option -port-as-server-id
- Add more info in
sb_includescript - Add connection info scripts to each sandbox
- Add cookbook scripts for replication between groups, ndb, master/slave.
- Fix "remote get" bug: it should fail on Mac (Issue #76)
- Fix port detection bug for NDB. The cluster port was not used in the search for free ports.
- Sort cookbook listing (it was being displayed in random order)
- Add test for replication between sandboxes (single and master/slave)
- Add option
--enable-admin-addressfor 8.0.14+ - When admin address is enabled,
./use_adminis created for each sandbox - multiple sandboxes have
./ma,./sa1./sa2,./na1,./na2anduse_all_admin
- Add command
cookbookwith subcommands list, show, create - Replace cookbook static sample scripts with dynamically created ones
- Add support for MySQL Cluster (topology=ndb)
- Add option
--ndb-nodesto replication
Change default authentication plugin for MariaDB 10.4.3 (Issue #67)
- Added support for Percona Xtradb Cluster (
--topology=pxc) [Linux only] - Added check for Unix tools before deploying.
- Added command
delete-binaries - Added option
--overwritetounpackcommand - Added flavors to
sandboxes - Added options
--flavorand--full-infotosandboxes - Added options
--flavor-in-prompttodeploy - Added option
--socket-in-datadirtodeploy - Added option
--prompttodeploy single
- Changed
remote getdocs to reflect changed names
- Fixed Issue #48 error with multiple plugins
- Fixed Issue #62 TiDB fails on MacOS
- Added script
after_startto sandboxes (does nothing by default, but it is used by TiDB to clean-up unwanted scripts)
-
Added support for TiDB
-
Added option
--client-from=X(Issue #49) -
Added option
--flavor -
Added flavor recording during unpack (Issue #52)
-
Added command
dbdeployer admin capabilities [flavor [version]] -
Added flavor detection based on files in expanded tarball (Issue #53)
Code improvements:
-
Version evaluation replaced by capabilities, i.e. flavor + version (Issue #50)
- Fix Issue #51 unpack command fails when tarball name doesn't include a version
- Fix unpack panic when tarball does not include top directory
- Fix count of total tests in
./test/all_tests.sh
- Added tests for TiDB, flavor detection, capabilities evaluation
- Fix Issue#46 - error creating catalog
- Some test improvements
- Added options
--read-only-slavesand--super-read-only-slaves todeploy replicationcommand. It only works for slaves ofmaster-slaveandfan-intopologies.
- Fixed bug in multi-master topologies, where replication ports were not calculated correctly
- Added check for IP address in replication creation
- Added test for read-only replication options
- Added common failure tests for sandbox creation
- Add support for remote tarballs.
- New commands:
remote listandremote download
- Removed all unconditional output in reusable code (common, sandbox)
- Changed many public functions to private when unused outside package.
- Added tests for most of functions in common
Minor bug fixes
- Changed all code that can Exit instead of returning an error in packages common, sandbox, defaults, unpack.
- Packages common and defaults keep the functions that can exit if they are only used with the CLI.
- Changed all functions returning errors to have the error as the last returning item.
- Moved constants and many global variables from 'defaults' to new 'globals' package. This reduce the risk of circular import between dbdeployer packages.
- Added function
IsEnvSet - Added variables for features minimum versions
- Added constants for sandbox script names
- Added constants for most used error messages
- Added comparison functions for testing in a separate package
- Replaced manual path composition with path.Join
- Added support for MySQL 4.1
- Added version info to sandbox scripts
- Added ability of unpacking "tar.xz" files (tarballs for Linux are compressed with xz instead of gzip as of MySQL 8.0.12)
- Fixed minor issue in unpack-shell. The unpacker was trying to move the shell extracted directory to the server directory.
- Code cleanup: fixed many code style violations
- Added unit test for abbreviations module.
- Merged in PR #39 (add port to prompt)
- Added tests for common/checks and common/tprintf
- Refactored some code to improve testability
- Replaced stack implementation
- Added tests for stack and concurrency
- Added cookbook scripts
- Fixed Issue #38 "--force does not check for locked sb"
- Added missing copyright on some files.
- Added check for missing copyright to
sanity_check.sh
- Added sandbox operations logging (on demand)
- Added option
--log-sb-operationsto enable logging - Added option
--log-directoryto change the default directory ($SANDBOX_HOME) - Added log-sb-operations and log-directory fields to default values.
- Added option
--repl-crash-safe(Issue #36) to get crash safe params without GTID.
- Fixed Issue #35 (--gtid should include relay-log-recovery)
- Fixed Issue #37 (slave initialization with GTID should use
MASTER_AUTO_POSITION) and added test for GTID behavior. - Some code refactoring: simplified exit procedure
- Code cleanup and formatting
- Moved build script to ./scripts
- Added option to build compressed executables (but defaults is still uncompressed)
- Added ./scripts/sanity_check.sh to verify formatting and code complaint.
- Fixed Issue#34 "dbdeployer fails when .mylogin.cnf is found"
- Fixed Issue#32 "Error log was not created in data directory"
- Added command 'dbdeployer admin upgrade sandbox1 sandbox2' (Issue#33)
- Implemented Issue#31 "Add feature to extract and merge a mysql-shell tarball into a regular one"
- Fixed Issue #30 "--rpl-user and --db-user should not allow root"
- Fixed Issue #26 "Port management glitch"
- Added ./vendor folder (simplifies dependencies)
- Fixed Issue #27 "undetected deps when building with MKDOCS" No binary releases for this issue, as only build.sh is affected.
- Improved naming of error log for each sandbox
- Implemented Issue 21 "Add support for directly using path to binaries"
- unpack would not act on old tarball where files were not explicitly marked as regular.
- Fixed Issue 22 "dbdeployer should check whether the binaries are for the current OS"
- Added test for Issue 21
- Added option for custom history file for each sandbox
- Added option for unified history file in replication sandboxes.
- Fixed bug in functional-test.sh, where some of the subtests were not executed.
- Improved error checking in all test scripts
Now it is possible to invoke dbdeployer with a short version number,
such as 5.7 or 8.0. In this case, dbdeployer will look for the latest
release of that MySQL version and use it.
- Command line was not saved to dbdeployer catalog
- Added test for short versions
- Added test for command line in catalog
- Fixed Issue #16 "Rename during unpack command fails" by making sure the path for sandbox-home and sandbox-binary are absolute ones.
- Added a stack of cleanup operations for group sandboxes when a depending sandbox installation fails
- Fixed help message of "unpack" command.
- When using
mysqld-debug, plugins are loaded from $BASEDIR/lib/plugin/debug (https://bugs.mysql.com/bug.php?id=89688)
- Added test for data dictionary tables exposure
- Improved documentation.
- Minor code refactoring
- Added option
--binary-version, which allows the basedir to be other than {prefix}x.x.xx - Added command line record to sbdescription.json, to the catalog, and to the defaults listing.
- Fixed Issue #10 (again). The directory for symlinks was not created timely, resulting in errors with some tarballs. VARIOUS
- Minor code refactoring
- Changes in templates for single sandbox scripts. Now almost all scripts source a file containing common code (sb_include). The behavior of the scripts is unchanged.
- Added test for
--binary-version
- Code reformatting and minor refactoring
- Fixed syntax error in code for tree.go
- Added coding sample minimal-sandbox2.go
- Merged pull request #11 from percona-csalguero/include_version_in_dir
- Fixed Issue #12 "deploying a sandbox with invalid version does not fail"
- Fixed minor bugs
- Removed unnecessary parameter in CreateSingleSandbox
- Added instructions to call CreateSingleSandbox from other apps See ./docs/coding
- Minor code refactoring for Exit calls
- Added mock sandbox creation for unit tests
- Added option
--enable-mysqlx(MySQL 5.7.12+) - Added options
--enable-general-logand--init-general-log - Added list of "reserverd-ports" to defaults
- Increased documentation inside command "usage"
- Added dbdeployer version and timestamp to sandbox descriprtion files.
- Added "mysqlsh" script to sandboxes 5.7.12+ with Xplugin enabled
- Added
show_logscript to all sandboxes - Improved interface of
show_binlogandshow_relaylog
- Added tests for reserved-ports
- Added test for mysqlsh and
show_*creation VARIOUS - Updated documentation
- Added support for mysqlx plugin being enabled by default (MySQL 8.0.11+)
- Added flag "--disable-mysqlx" to disable mysqlx plugin (8.0.11+)
- Added scripts
use_all_mastersanduse_all_slavesto all replication sandboxes. - Added option
--verbosity={0,1,2}to unpack command.
- Fixed Issue#10 "dbdeployer unpack does not handle symlinks"
- Fixed minor bug in documentation test builder.
- Added tests for number of ports, log errors,
use_all_masters,use_all_slaves, running processes. - Added options to stop tests after a given set of operations.
- Removed restriction on running 5.6 tests in docker for Mac.
- Added option
--skip-start - Added report-port and report-host automatically to my.sandbox.cnf
- Added options
--skip-report-hostand--skip-report-port - Added documentation dbdeployer compiling.
- Added documentation for
--skip-start - Enhanced build.sh to handle dependencies.
- Added tests for
--skip-startand report-host/report-port behavior.
- Added more documentation
- Added bash-completion script
- Moved hidden command "tree" to conditional compiling. Now there is a separated build for docs-enabled dbdeployer
- Added ability of producing more documentation using command "tree"
- Added ability of handling environment variables in configuration file. $HOME and $PWD are expanded to actual values.
- Added hidden command "tree" that can generate the full dbdeployer API. Using this feature, from now on we can compare API changes automatically.
- Fixed visualization of sandboxes from catalog
- Fixed minor code issues.
- Added tests for environment variable replacement
- Fixed Issue #5 "Single deployment doesn't show the location of the sandbox"
- Added API definition (
./docs/API-1.0.md) - Added test for Issue #5
- Fixed typos and improved docs.
- General Availability.
- Fixed bug with single deployment and --force. On the second deployment, the port was changed.
- More tests added. The test suite now runs a total of 3,013 tests (MacOS) and 3,143 (Linux). A total of 6,156 tests that ran at least twice (once with concurrency and once without)
- Added version detection to
unpackcommand. Now--unpack-versionbecomes mandatory only if a version is not detected from the tarball name. - Added
--headerflag tosandboxescommand. - More tests and improved tests.
- Fixed deployment bug in fan-in replication
- Added tests for fan-in replication, sandbox completeness, start,
restart, and
add_options.
- Added
--semi-syncoption to replication - Added more tests
- Minor change to templates
- Added test for export/import templates
- Added more tests for pre/post grants SQL
- Added test for on-the-fly template replacement
- Trivial changes to "sandboxes" output
- Changed test for group replication (now uses the same defined for multi-source replication)
- Improved usability of tests.
- Made tests easier to extend.
- Added test for pre/post grants SQL.
- Added (mock) tests for unpack command
- Improved test reporting
- Added list of environment variables
- Minor bug fixes
- Added more tests
- Added topologies "fan-in" and "all-masters"
- Feature complete: This is release candidate for 1.0
- Fixed bug on UUID generation.
- Implemented parallel deployment of multiple sandboxes
- Flag --concurrent is available for deploy and delete
- Improved tests
- Added --catalog to "sandboxes" command
- Improved tests
- INCOMPATIBLE CHANGES:
- MySQL 8.0.x now starts with
caching_sha2_passwordby default. - flag "
--keep-auth-plugin" was removed. Instead, we now have "--native-auth-plugin", false by default, which will use the old plugin if enabled. - The sandbox catalog is now enabled by default. It can be disabled
using either the environment variable
SKIP_DBDEPLOYER_CATALOGor using the configuration file.
- MySQL 8.0.x now starts with
- Added workaround for bug#89959: replication with 8.0 and
caching_sha2_passwordfails
- Improved mock test speed by parametrizing sleep intervals:
- 200 mock sandboxes tested in 73 seconds (previously, 15 minutes).
- 2144 mock sandboxes tested in 23 minutes (previously, 145 minutes)
- Now dbdeployer finds unused ports automatically, to avoid conflicts.
- Added ability of running faster tests with mock MySQL packages.
- Added
--defaultsflag - Removed hardcoded names for multiple sandbox directories and shortcuts.
- Added directory names and shortcuts for multiple sandboxes to configuration data
- Added ability of exporting/importing a single template.
- Fixed visualization error with template export
- Added interactivity to main test script.
-
INCOMPATIBLE CHANGES:
- "single", "multiple", and "replication" are now subcommands of "deploy".
- Previous "admin" commands are now under "defaults"
- "templates" is now a subcommand of "defaults"
- New "admin" command only supports "lock" and "unlock"
-
EXPERIMENTAL FEATURE: There is a sandbox catalog being created and updated in
$HOME/.dbdeployer/sandboxes.json. The deployment and deletion commands handle the catalog transparently. Disabled by default. It can be enabled by setting the environment variableDBDEPLOYER_CATALOG
- Added commands "admin lock" and "admin unlock" to prevent/allow deletion of a sandbox.
- Added placeholder fields for multi-source clustering in defaults
- Fixed bug with "sandboxes" command. It would not check if the
sandbox_homedirectory existed. - Fixed bug in "sandboxes" command. It would not report sandboxes created by other applications (MySQL-Sandbox)
- Added check for template version during export/import
- Added tests for UUID generation
- Improved docker test
- Added "test-replication" to "global" command
- Added several aliases to "unpack"
- Changed template
init_db, to allow for easier customization - Added test for docker. The full test suite can run in a container.
- Simplified test.sh by using "dbdeployer global" rather than hand made loops.
- All values used for sandbox deployment are now modifiable.
- Added command "admin" to deal with default values: show, store, remove, load, update, export
- Refactored global variables to become modifiable through the "admin" command
- Added commands for template export and import.
- Added flag
--expose-dd-tablesto show data dictionary hidden tables - Added flag
--custom-mysqldto use different server executable
- Added flags for pre and post grants SQL execution.
--pre-grants-sql-file--pre-grants-sql--post-grants-sql-file--post-grants-sql- Fixed bug (in cobra+pflag package) that splits multiple commands by comma.
- MySQL 8.0+ sandboxes now use roles instead of direct privilege assignments.
- Added global flag --force to overwrite an existing deployment
- Added global flag
--my-cnf-fileto use a customized my.cnf - Added flag
--master-ipto replication deployments - Fixed bug in abbreviations: flags were not considered correctly.
- The "delete" command now supports "ALL" as argument. It will delete all installed sandboxes.
- Added flag "--skip-confirm" for the "delete" command, to delete without confirmation.
- Fixed mistake during initialization: the version search was happening before the check for the sandbox home directory.
- Added the "global" command to propagate a command to all sandboxes
- Added automated README generation
- minor code changes
- Added automatic generation of human-readable server-UUID
- Added flag
--keep-server-uuidto prevent the above change
- Changed default port and sandbox directory for single-primary group replication.
- Added custom abbreviations feature.
- Added script
test_sbto every single sandbox - Added script
test_sb_allto all multiple/group/replication sandbox - Added script
test_replicationto replication sandboxes - Added test/test.sh, which runs a comprehensive test of most dbdeployer features
- Added command "templates export"
- Added flag
--single-primaryfor group replication - Added flags
--sandbox-directory, --port, and base-port to allow deploying several sandboxes of the same version. - Added a check for clash on installed ports
- INCOMPATIBLE change: Changed format of sbdescription.json: now can list several ports per sandbox.
- Added a check for version before applying gtid.
- Added commands templates list/show/describe
- Added
--use-template=template_name:file_nameflag
- Improved check for tarball as an argument to single, replication, multiple.
- Improved help for single, multiple, and replication
- Added customized prompt for configuration file
- Changed initialization method to use tarball libraries
- Fixed glitch in "unpack" when original tarball has clashing name
- Updated README.md
- Changed formatting for "usage" command
- Run detection of invalid group replication earlier.
- Added version to sandbox description file
- Added sandbox description file
- 'sandboxes' command uses above file for sandbox listing
- Added 'delete' command
- improved documentation
- Added "usage" command
- Added description to "sandboxes" output
- Added check for version format
- Changed message for missing argument
- Added check for sandbox-home existence
- Added group replication topology.
- Added option --master to 'single' command
- Added new commands to each sandbox:
add_option,show_binlog,show_relaylog, my.
- Added short names for some flags.
- Improved commands usage text
- Modified
--my-cnf-optionsand --init-options to be accepted multiple times
- Fixed bug in unpack when basedir was not created.
- Fixed inclusion of options in my.sandbox.cnf (
--my-cnf-options) - Added command 'multiple'
- Enhanced documentation
- Initial commit with basic features migrated from MySQL-Sandbox