Commit Graph

656 Commits

Author SHA1 Message Date
Sergei Golubchik
3da761912a MDEV-6616 Server crashes in my_hash_first if shutdown is performed when FLUSH LOGS is running
master_info_index becomes zero during shutdown.
check that it's valid (under a mutex) before dereferencing.
2014-09-06 08:33:56 +02:00
Jan Lindström
df4dd593f2 MDEV-6247: Merge 10.0-galera to 10.1.
Merged lp:maria/maria-10.0-galera up to revision 3879.

Added a new functions to handler API to forcefully abort_transaction,
producing fake_trx_id, get_checkpoint and set_checkpoint for XA. These
were added for future possiblity to add more storage engines that
could use galera replication.
2014-08-26 15:43:46 +03:00
Sergei Golubchik
6fb17a0601 5.5.39 merge 2014-08-07 18:06:56 +02:00
Sergei Golubchik
1c6ad62a26 mysql-5.5.39 merge
~40% bugfixed(*) applied
~40$ bugfixed reverted (incorrect or we're not buggy)
~20% bugfixed applied, despite us being not buggy
(*) only changes in the server code, e.g. not cmakefiles
2014-08-02 21:26:16 +02:00
Kristian Nielsen
9150a0c7cb MDEV-4937: sql_slave_skip_counter does not work with GTID
The sql_slave_skip_counter is important to be able to recover replication from
certain errors. Often, an appropriate solution is to set
sql_slave_skip_counter to skip over a problem event. But setting
sql_slave_skip_counter produced an error in GTID mode, with a suggestion to
instead set @@gtid_slave_pos to point past the problem event. This however is
not always possible; for example, in case of an INCIDENT event, that event
does not have any GTID to assign to @@gtid_slave_pos.

With this patch, sql_slave_skip_counter now works in GTID mode the same was as
in non-GTID mode. When set, that many initial events are skipped when the SQL
thread starts, plus as many extra events are needed to completely skip any
partially skipped event group. The GTID position is updated to point past the
skipped event(s).
2014-06-25 15:24:11 +02:00
Venkatesh Duggirala
2870bd7423 Bug#17283409 4-WAY DEADLOCK: ZOMBIES, PURGING BINLOGS,
SHOW PROCESSLIST, SHOW BINLOGS

Problem:  A deadlock was occurring when 4 threads were
involved in acquiring locks in the following way
Thread 1: Dump thread ( Slave is reconnecting, so on
              Master, a new dump thread is trying kill
              zombie dump threads. It acquired thread's
              LOCK_thd_data and it is about to acquire
              mysys_var->current_mutex ( which LOCK_log)
Thread 2: Application thread is executing show binlogs and
               acquired LOCK_log and it is about to acquire
               LOCK_index.
Thread 3: Application thread is executing Purge binary logs
               and acquired LOCK_index and it is about to
               acquire LOCK_thread_count.
Thread 4: Application thread is executing show processlist
               and acquired LOCK_thread_count and it is
               about to acquire zombie dump thread's
               LOCK_thd_data.
Deadlock Cycle:
     Thread 1 -> Thread 2 -> Thread 3-> Thread 4 ->Thread 1

The same above deadlock was observed even when thread 4 is
executing 'SELECT * FROM information_schema.processlist' command and
acquired LOCK_thread_count and it is about to acquire zombie
dump thread's LOCK_thd_data.

Analysis:
There are four locks involved in the deadlock.  LOCK_log,
LOCK_thread_count, LOCK_index and LOCK_thd_data.
LOCK_log, LOCK_thread_count, LOCK_index are global mutexes
where as LOCK_thd_data is local to a thread.
We can divide these four locks in two groups.
Group 1 consists of LOCK_log and LOCK_index and the order
should be LOCK_log followed by LOCK_index.
Group 2 consists of other two mutexes
LOCK_thread_count, LOCK_thd_data and the order should
be LOCK_thread_count followed by LOCK_thd_data.
Unfortunately, there is no specific predefined lock order defined
to follow in the MySQL system when it comes to locks across these
two groups. In the above problematic example,
there is no problem in the way we are acquiring the locks
if you see each thread individually.
But If you combine all 4 threads, they end up in a deadlock.

Fix: 
Since everything seems to be fine in the way threads are taking locks,
In this patch We are changing the duration of the locks in Thread 4
to break the deadlock. i.e., before the patch, Thread 4
('show processlist' command) mysqld_list_processes()
function acquires LOCK_thread_count for the complete duration
of the function and it also acquires/releases
each thread's LOCK_thd_data.

LOCK_thread_count is used to protect addition and
deletion of threads in global threads list. While show
process list is looping through all the existing threads,
it will be a problem if a thread is exited but there is no problem
if a new thread is added to the system. Hence a new mutex is
introduced "LOCK_thd_remove" which will protect deletion
of a thread from global threads list. All threads which are
getting exited should acquire LOCK_thd_remove
followed by LOCK_thread_count. (It should take LOCK_thread_count
also because other places of the code still thinks that exit thread
is protected with LOCK_thread_count. In this fix, we are changing
only 'show process list' query logic )
(Eg: unlink_thd logic will be protected with
LOCK_thd_remove).

Logic of mysqld_list_processes(or file_schema_processlist)
will now be protected with 'LOCK_thd_remove' instead of
'LOCK_thread_count'.

Now the new locking order after this patch is:
LOCK_thd_remove -> LOCK_thd_data -> LOCK_log ->
LOCK_index -> LOCK_thread_count
2014-05-08 18:13:01 +05:30
unknown
2c2478b822 MDEV-5804: If same GTID is received on multiple master connections in multi-source replication, the event is double-executed causing corruption or replication failure
Before, the arrival of same GTID twice in multi-source replication
would cause double-apply or in gtid strict mode an error.

Keep the behaviour, but add an option --gtid-ignore-duplicates which
allows to correctly handle duplicates, ignoring all but the first.
This relies on the user ensuring correct configuration so that
sequence numbers are strictly increasing within each replication
domain; then duplicates can be detected simply by comparing the
sequence numbers against what is already applied.

Only one master connection (but possibly multiple parallel worker
threads within that connection) is allowed to apply events within
one replication domain at a time; any other connection that
receives a GTID in the same domain either discards it (if it is
already applied) or waits for the other connection to not have
any events to apply.

Intermediate patch, as proof-of-concept for testing. The main limitation
is that currently it is only implemented for parallel replication,
@@slave_parallel_threads > 0.
2014-03-09 10:27:38 +01:00
Sergei Golubchik
0dc23679c8 10.0-base merge 2014-02-26 15:28:07 +01:00
Sergei Golubchik
0b9a0a3517 5.5 merge 2014-02-25 16:04:35 +01:00
Sergei Golubchik
84651126c0 MySQL-5.5.36 merge
(without few incorrect bugfixes and with 1250 files where only a copyright year was changed)
2014-02-17 11:00:51 +01:00
unknown
dd93ec5633 Merge MariaDB 10.0-base to 10.0. 2014-02-10 15:12:17 +01:00
unknown
4e6606acad MDEV-4984: Implement MASTER_GTID_WAIT() and @@LAST_GTID.
MASTER_GTID_WAIT() is similar to MASTER_POS_WAIT(), but works with a
GTID position rather than an old-style filename/offset.

@@LAST_GTID gives the GTID assigned to the last transaction written
into the binlog.

Together, the two can be used by applications to obtain the GTID of
an update on the master, and then do a MASTER_GTID_WAIT() for that
position on any read slave where it is important to get results that
are caught up with the master at least to the point of the update.

The implementation of MASTER_GTID_WAIT() is implemented in a way
that tries to minimise the performance impact on the SQL threads,
even in the presense of many waiters on single GTID positions (as
from @@LAST_GTID).
2014-02-07 19:15:28 +01:00
Michael Widenius
0a20d762af Fix for MDEV-4117 @@global.relay_log_purge not per-master, conflicts between different masters in multisource replication
The fix is to not change @relay_log_purge as part of the CHANGE MASTER.
(There is no logical reason why this is done in the current source)

mysql-test/suite/rpl/r/rpl_slave_status.result:
  Ensure that CHANGE MASTER doesn't change relay_log_purge
mysql-test/suite/rpl/t/rpl_slave_status.test:
  Ensure that CHANGE MASTER doesn't change relay_log_purge
sql/sql_repl.cc:
  Don't change relay_log_purge in CHANGE MASTER
2014-01-14 19:00:38 +01:00
Sergei Golubchik
d28d3ba40d 10.0-base merge 2013-12-16 13:02:21 +01:00
unknown
170e9e593d MDEV-5306: Missing locking around rpl_global_gtid_binlog_state
There were some places where insufficient locking between
parallel threads could cause invalid memory accesses and
possibly other grief.

This patch adds the missing locking, and moves the locking
into the struct rpl_binlog_state methods to make it easier
to see that proper locking is in place everywhere.
2013-11-18 15:22:50 +01:00
Venkatesh Duggirala
e0efc2c39a Bug#17641586 INCORRECTLY PRINTED BINLOG DUMP INFORMATION
Problem:
When log_warnings is greater than 1, master prints binlog
dump thread information in mysqld.1.err file.
The information contains slave server id, binlog file and
binlog position. The slave server id is uint32 and the print
format was wrongly specifified (%d instead of %u).
Hence a server id which is more than 2 billion is getting
printed with a negative value.
Eg: Start binlog_dump to slave_server(-1340259414),
pos(mysql-bin.001663, 325187493)

Fix: Changed the uint32 format to %u.
2013-11-12 22:09:10 +05:30
Sergei Golubchik
9af177042e 10.0-base merge.
Partitioning/InnoDB changes are *not* merged (they'll come from 5.6)
TokuDB does not compile (not updated to 10.0 SE API)
2013-09-21 10:14:42 +02:00
unknown
ada15c7a0f Fix various places where code would work incorrectly if the common_header_len of events is different on master and slave
Patch developed with the help of Pavel Ivanov.

Also fix an uninitialised variable in queue_event().
2013-09-04 12:22:09 +02:00
unknown
f9c2b402f4 MDEV-26: Global transaction ID.
Implement @@gtid_binlog_state. This is the internal state of the binlog
(most recent GTID logged for every domain_id and server_id). This allows
to save the state before RESET MASTER and restore it afterwards.
2013-08-23 14:02:13 +02:00
unknown
f0deff867a MDEV-4820: Empty master does not give error for slave GTID position that does not exist in the binlog
The main bug here was the following situation:

Suppose we set up a completely new master2 as an extra multi-master to an
existing slave that already has a different master1 for domain_id=0. When the
slave tries to connect to master2, master2 will not have anything that slave
requests in domain_id=0, but that is fine as master2 is supposedly meant to
serve eg. domain_id=1. (This is MDEV-4485).

But suppose that master2 then actually starts sending events from
domain_id=0. In this case, the fix for MDEV-4485 was incomplete, and the code
would fail to give the error that the position requested by the slave in
domain_id=0 was missing from the binlogs of master2. This could lead to lost
events or completely wrong replication.

The patch for this bug fixes this issue.

In addition, it cleans up the code a bit, getting rid of the fake_gtid_hash in
the code. And the error message when slave and master have diverged due to
alternate future is clarified, as requested in the bug description.
2013-08-16 15:10:25 +02:00
Sergei Golubchik
b7b5f6f1ab 10.0-monty merge
includes:
* remove some remnants of "Bug#14521864: MYSQL 5.1 TO 5.5 BUGS PARTITIONING"
* introduce LOCK_share, now LOCK_ha_data is strictly for engines
* rea_create_table() always creates .par file (even in "frm-only" mode)
* fix a 5.6 bug, temp file leak on dummy ALTER TABLE
2013-07-21 16:39:19 +02:00
Sergei Golubchik
5f6380adde 10.0-base merge 2013-07-18 16:46:57 +02:00
Sergei Golubchik
97e640b9ae 5.5 merge 2013-07-17 21:24:29 +02:00
Sergei Golubchik
005c7e5421 mysql-5.5.32 merge 2013-07-16 19:09:54 +02:00
unknown
2f6a2494a5 MDEV-4708: GTID strict mode doesn't work on a database with purged binlogs
When a new master is provisioned that does not have any old binlogs,
the @@gtid_slave_pos is used to know where in the GTID history the
provisioning happened. A slave is allowed to connect at the point of
this value of @@gtid_slave_pos, even if that GTID is not in the
binlogs on the new master.

The code to handle this case when the binlog on the newly provisioned
master is completely empty was just wrong (couple of typos). Clearly it
had never been tested ... :-/
2013-07-10 12:01:52 +02:00
unknown
1e43277838 MDEV-4708: GTID strict mode doesn't work on a database with purged binlogs
When a new master is provisioned that does not have any old binlogs,
the @@gtid_slave_pos is used to know where in the GTID history the
provisioning happened. A slave is allowed to connect at the point of
this value of @@gtid_slave_pos, even if that GTID is not in the
binlogs on the new master.

But --gtid-strict-mode did not correctly handle this case. When strict
mode was enabled, an attempt to connect at the position would cause an
error about holes in the binlog, which is not correct.

This patch adds a hash of GTIDs that need to be treated specially by
GTID strict mode to deal correctly with this case.
2013-07-10 11:45:15 +02:00
Michael Widenius
5f1f2fc0e4 Applied all changes from Igor and Sanja 2013-06-15 18:32:08 +03:00
unknown
b5fcf33d24 MDEV-4490: Old-style master position points at the last GTID event after slave restart
Now whenever we reach the GTID point requested from the slave (when using GTID
position to connect), we send a fake Gtid_list event. This event is used by
the slave to know the current old-style position for MASTER_POS_WAIT(), and
later the similar binlog position for MASTER_GTID_WAIT().

Without this fake event, if the slave is already fully up-to-date with the
master, there may be no events sent at the given position for an indeterminate
time.
2013-06-07 14:39:00 +02:00
unknown
7b6ab5638a MDEV-4483: CHANGE MASTER TO master_use_gtid=xxx looses old-style coordinates.
There was some old code that cleared the position in CHANGE MASTER,
it was forgotten to be removed.

In addition, add code that saves/restores the old-style position
when we nuke the old relay logs as part of GTID slave start.
Normally we will not use these, but it could be useful in case
the GTID connect fails and user wants to go back to the old-style
coordinates.
2013-06-07 08:43:21 +02:00
Sergei Golubchik
72ba95873a 10.0-base merge
(without InnoDB - all InnoDB changes were ignored)
2013-06-06 21:32:29 +02:00
Sergei Golubchik
4749d40c63 5.5 merge 2013-06-06 17:51:28 +02:00
unknown
5cb486d159 MDEV-26: Global transaction ID.
Fix problems related to reconnect. When we need to reconnect (ie. explict
stop/start of just the IO thread by user, or automatic reconnect due to
loosing network connection with the master), it is a bit complex to correctly
resume at the right point without causing duplicate or missing events in the
relay log. The previous code had multiple problems in this regard.

With this patch, the problem is solved as follows. The IO thread keeps track
(in memory) of which GTID was last queued to the relay log. If it needs to
reconnect, it resumes at that GTID position. It also counts number of events
received within the last, possibly partial, event group, and skips the same
number of events after a reconnect, so that events already enqueued before the
reconnect are not duplicated.

(There is no need to keep any persistent state; whenever we restart slave
threads after both of them being stopped (such as after server restart), we
erase the relay logs and start over from the last GTID applied by SQL thread.
But while the SQL thread is running, this patch is needed to get correct relay
log).
2013-06-05 14:32:47 +02:00
unknown
385780f571 MDEV-4485: Master did not allow slave to connect from the very start (empty GTID pos) if GTIDs from other multi_source master was present
The idea in the code was to protect the user that tries to connect a slave
to a master with completely different domains than what was intended. If
none of the domains in the start position are present at all in the master
binlog, we gave an error.

However, this is a stupid idea. Because when a slave connects to a master
to start replication from the very start of binlogs - such as when setting
up new master->slave servers from scratch - there will be just this
situation, the requested slave position is empty for all the domains in the
master's binlog.

So the code that gives this error is wrong, and the solution is simply to
remove it.
2013-05-29 11:41:25 +02:00
unknown
a0fd7382bc Merge 10.0-base -> 10.0 2013-05-28 15:39:56 +02:00
unknown
ee2b7db3f8 MDEV-4478: Implement GTID "strict mode"
When @@GLOBAL.gtid_strict_mode=1, then certain operations result
in error that would otherwise result in out-of-order binlog files
between servers.

GTID sequence numbers are now allocated independently per domain;
this results in less/no holes in GTID sequences, increasing the
likelyhood that diverging binlogs will be caught by the slave when
GTID strict mode is enabled.
2013-05-28 13:28:31 +02:00
unknown
1cd6eb5f94 MDEV-26: Global transaction ID.
Change of user interface to be more logical and more in line with expectations
to work similar to old-style replication.

User can now explicitly choose in CHANGE MASTER whether binlog position is
taken into account (master_gtid_pos=current_pos) or not (master_gtid_pos=
slave_pos) when slave connects to master.

@@gtid_pos is replaced by three separate variables @@gtid_slave_pos (can
be set by user, replicated GTIDs only), @@gtid_binlog_pos (read only), and
@@gtid_current_pos (a combination of the two, most recent GTID within each
domain). mysql.rpl_slave_state is renamed to mysql.gtid_slave_pos to match.

This fixes MDEV-4474.
2013-05-22 17:36:48 +02:00
unknown
d795bc9ff8 Fix race condition in binlog dump thread during server shutdown.
There was missing a check for THD::killed after THD::enter_cond(). This could
cause the binlog dump thread to miss the kill signal during server shutdown
and hang until it was force-closed.

Also fix a race in a test case that occasionally fails in Buildbot.
2013-05-16 12:41:11 +02:00
unknown
9fae993024 MDEV-26: Global transaction ID.
Implement START SLAVE UNTIL master_gtid_pos = "<GTID position>".

Add test cases, including a test showing how to use this to promote
a new master among a set of slaves.
2013-05-15 19:52:21 +02:00
Sergei Golubchik
b381cf843c mysql-5.5.31 merge 2013-05-07 13:05:09 +02:00
unknown
d0d05dae07 Merge 10.0-base -> 10.0 2013-05-03 12:10:16 +02:00
unknown
5aa0d185ca MDEV-4473: mysql_binlog_send() starts sending events from wrong GTID position in some master failover scenarios
Suppose binlog file X has in its Gtid_list_event: 0-1-3,0-2-5, and suppose the
slave requests to start replicating after 0-1-3.

In this case the bug was that master would start sending events from the start
of X. This is wrong, because 0-2-4 and 0-2-5 are contained in X-1, and are
needed by the slave. So these events were lost.

On the other hand, if the slave requested 0-2-5, then it _is_ correct to start
sending from the beginning of binlog file X, because 0-2-5 is the last GTID
logged in earlier binlogs. The difference is that 0-2-5 is the last of the
GTIDs in the Gtid_list_event. The problem was that the code did not check that
the matched GTID was the last one in the list.

Fixed by checking if the gtid requested by slave that matches a gtid in the
Gtid_list_event is the last event for that domain in the list. If not, go back
to a prior binlog to ensure all needed events are sent to slave.

mysql-test/include/show_events.inc:
  Backport --let $binlog_file=LAST, used by MDEV-4473 test case.
2013-05-03 11:27:29 +02:00
Michael Widenius
8cdb118a0a Fixed: MDEV-4352; LOAD DATA was not multi-source safe
- Calls to cleanup_load_tmpdir() could delete temporary files for another master connection
- Concurrent LOAD DATA commands from two master connections could use the same file name

Other bug fixes:
- Enlarge buffer for connection names with 'special characters' one can't store in filenames

Optimization:
- Don't do 'lower case' of connection names. We can use cmp_connection_name, where we already have the connection name in lower case.


mysql-test/suite/multi_source/load_data.result:
  Test case for MDEV-4352
mysql-test/suite/multi_source/load_data.test:
  Test case for MDEV-4352
sql/log_event.cc:
  Fixed: MDEV-4352
  - Calls to cleanup_load_tmpdir() could delete temporary files for another master connection
  - Concurrent LOAD DATA commands from two master connections could use the same file name
  
  The fix was to add the connection name (if one exists) to all slave temporary files used by LOAD DATA
sql/rpl_mi.cc:
  Enlarge buffer for connection names with 'special characters' one can't store in filenames
  Use mi->cmp_connection_name for connection file names.
sql/rpl_rli.cc:
  Use mi->cmp_connection_name for connection file names.
sql/slave.cc:
  Removed not needed empty line
sql/sql_const.h:
  Added MAX_FILENAME_MBWIDTH to be able to calculate buffer length for connection_names stored in file names
sql/sql_repl.cc:
  Use mi->cmp_connection_name for connection file names.
2013-05-03 01:50:42 +03:00
unknown
56d485e2b5 Merge 10.0-base -> 10.0 2013-04-29 12:03:54 +02:00
unknown
59830e1ab8 MDEV-4446: Incorrect handling of binlog checksum when searching for GTID start position in binlog
When the slave connects, the master skips binlog event groups
until it reaches the position requested by the slave. To
identify event groups, it needs to detect COMMIT events. But
this detection did not correctly handle binlog checksums, so
could incorrectly skip extra groups due to not detecting the
end of an event group.
2013-04-29 10:57:48 +02:00
unknown
203264ddc9 Fix unsigned/signed conversion bug in event type during mysql_binlog_send().
Since event types can be >=128 and are read from a (possibly signed) char
pointer, we need to cast to unsigned char before extending to int, or we will
get an incorrect negative number. This was done in the main code path already,
but there is a rare case where we check for new events first without a lock
and then again with the lock. If the second check succeeds because a new event
turns up at just the right time, then we took a code path that was missing the
correct unsigned char cast, leading to incorrect handling of events for old
slave servers and possibly other grief.

(This was found from a sporadic failure in Buildbot of test case
rpl_mariadb_slave_capability).
2013-04-25 13:16:35 +02:00
unknown
6b97512b21 Add missing check for thd->killed in mysql_binlog_send().
The slave dump thread running on the master only checked thd->killed whenever
it reached the end of a binlog file, not between events. This could
unnecessarily delay server shutdown.

This was found by code inspection while tracking down some occasional "forcing
close of thread..." errors in Buildbot. Hopefully this will fix the failures,
but the fix is correct in any case.

Also increase the wait during server shutdown, 2 seconds is a bit tight in
case of heavy I/O stall, and it seems better to delay shutdown a bit than
force-kill threads unnecessarily.

Also fix some races in test cases that restart the mysqld server. The .expect
file should be changed with --append_file, --remove_file + --write_file
creates a short window where mysqld can error out due to .expect file missing.
2013-04-24 13:05:40 +02:00
Sergei Golubchik
84ce6832e6 MDEV-4332 Increase username length from 16 characters 2013-04-18 22:17:29 +02:00
Sergei Golubchik
07315d3603 strmake_buf(X,Y) helper, equivalent to strmake(X,Y,sizeof(X)-1)
with a bit of lame protection against abuse.
2013-04-17 19:42:34 +02:00
unknown
0e7410a154 Merge 10.0-base -> 10.0 (GTID). 2013-04-17 15:17:01 +02:00
Sergei Golubchik
a9035be5b7 10.0-base merge 2013-04-15 15:09:22 +02:00
unknown
665a31af2b MDEV-26: Global transaction ID. First alpha release.
Merge of 10.0-mdev26 feature tree into 10.0-base.

Global transaction ID is prepended to each event group in the binlog.

Slave connect can request to start from GTID position instead of specifying
file name/offset of master binlog. This facilitates easy switch to a new
master.

Slave GTID state is stored in a table mysql.rpl_slave_state, which can be
InnoDB to get crash-safe slave state.

GTID includes a replication domain ID, allowing to keep track of distinct
positions for each of multiple masters.
2013-04-15 10:55:27 +02:00
Sergei Golubchik
4ec6fe10e5 remove ULL() and LL(), because they're totally unnecessary
and sometimes harmful (used with expressions)
2013-04-07 14:00:16 +02:00
unknown
b7363eb4ac MDEV-26: Global transaction ID.
Replace CHANGE MASTER TO ... master_gtid_pos='xxx' with a new system
variable @@global.gtid_pos.

This is more logical; @@gtid_pos is global, not per-master, and it is not
affected by RESET SLAVE.

Also rename master_gtid_pos=AUTO to master_use_gtid=1, which again is more
logical.
2013-04-05 16:20:58 +02:00
unknown
c2cbc9cee6 MDEV-26: Global transaction ID.
Move combining slave and gtid binlog state into a separate function.

Make SHOW ALL SLAVES STATUS use the same function, so it shows the
same value used by slave connect.

Add a test case.
2013-04-04 17:38:10 +02:00
Sujatha Sivakumar
9a1d651b0e Merge from mysql-5.1 to mysql-5.5 2013-03-28 14:18:51 +05:30
Sujatha Sivakumar
d054027c4b Bug#14324766:PARTIALLY WRITTEN INSERT STATEMENT IN BINLOG
NO ERRORS REPORTED
      
Problem:
=======
Errors from my_b_fill are ignored. MYSQL_BIN_LOG::write_cache
code assumes that 0 returned from my_b_fill always means
end-of-cache, but that is incorrect. It can result in error
and the error is ignored. Other callers of my_b_fill don't
check for error: my_b_copy_to_file, maybe my_b_gets.
      
Fix:
===
An error handler is already present to check the "cache"
error that is reported during "MYSQL_BIN_LOG::write_cache"
call. Hence error handlers are added for "my_b_copy_to_file"
and "my_b_gets".
During my_b_fill() function call, when the cache read fails
info->error= -1 is set. Hence a check for "info->error"
is added for the above to callers upon their return.

mysys/mf_iocache2.c:
  Added a check for "cache->error" and simulation of cache read failure
mysys/my_read.c:
  Simulation of read failure
sql/log_event.cc:
  Added debug simulation
sql/sql_repl.cc:
  Added a check for cache error
2013-03-28 14:14:39 +05:30
Sergei Golubchik
993ea79f2d 5.5 merge 2013-03-27 23:41:02 +01:00
unknown
0fdbdde474 MDEV-26: Global transaction ID.
Implement test case rpl_gtid_stop_start.test to test normal stop and restart
of master and slave mysqld servers.

Fix a couple bugs found with the test:

 - When InnoDB is disabled (no XA), the binlog state was not read when master
   mysqld starts.

 - Remove old code that puts a bogus D-S-0 into the initial binlog state, it
   is not correct in current design.

 - Fix memory leak in gtid_find_binlog_file().
2013-03-27 16:06:45 +01:00
unknown
cb65cee82a MDEV-26: Global transaction ID.
When slave requested to start at some GTID, and that GTID was the very
last event (within its replication domain) in some binlog file, we did
not allow the binlog dump thread on the master to start from the
beginning of a following binlog file. This is a problem, since the
binlog file containing the GTID is likely to be purged if the
replication domain is unused for long.

With this fix, if the Gtid list event at the start of a binlog file
contains exactly the GTID requested by the slave, we allow to start
the binlog dump thread from this file, taking care not to skip any
events from that domain in the file.
2013-03-27 12:29:02 +01:00
unknown
20e021115f MDEV-26: Global transaction ID.
Fix MDEV-4329. When user does CHANGE MASTER TO
MASTER_GTID_POS='<explicit GTID state>', we check that this state
does not conflict with the binlog. But the code forgot to give an
error in the case where a domain was completely missing from the
requested position (eg. MASTER_GTID_POS='').
2013-03-27 09:37:54 +01:00
unknown
f66e3474a8 MDEV-26: Global transaction ID.
Fix missing error check for applying Gtid_log_event.

Fix a couple compiler warnings.
2013-03-26 14:58:14 +01:00
Michael Widenius
068c61978e Temporary commit of 10.0-merge 2013-03-26 00:03:13 +02:00
unknown
a3f651d0f6 MDEV-26: Global transaction ID.
Fix checksums for the new GTID events (reported by Pavel Ivanov).
Fix printing of GTID_LIST_EVENT.
2013-03-22 11:26:28 +01:00
unknown
e590f89114 MDEV-26: Global transaction ID.
Fix MDEV-4278: Slave does not check that master understands GTID.

Now the slave will abort with a suitable error if an attempt is made to connect
with GTID to a master that does not support GTID.
2013-03-21 12:16:04 +01:00
unknown
9bb989a9d1 MDEV-26: Global transaction ID.
Fix MDEV-4275 - I/O thread restart duplicates events in the relay log.
The first time we connect to master after CHANGE MASTER or restart, we connect
from the GTID position. But then subsequent reconnects or IO thread restarts
reconnect with the old-style file/offset binlog pos from where it left off at
last disconnect. This is necessary to avoid duplicate events in the relay
logs, as there is nothing that synchronises the SQL thread update of GTID
state (multiple threads in case of multi-source) with IO thread reconnects.

Test cases.

Some small cleanups and fixes.
2013-03-21 11:03:31 +01:00
unknown
9d9ddad759 MDEV-26: Global transaction ID.
Fix things so that a master can switch with MASTER_GTID_POS=AUTO to a slave
that was previously running with log_slave_updates=0, by looking into the
slave replication state on the master when the slave requests something not
present in the binlog.

Be a bit more strict about what position the slave can ask for, to avoid some
easy-to-hit misconfiguration errors.

Start over with seq_no counter when RESET MASTER.
2013-03-18 15:09:36 +01:00
unknown
379819d8c0 MDEV-26: Global transaction ID.
After-rebase fixes. Rebase was done to be able to build on top
of the binlog background thread of MDEV-532.
2013-03-14 10:39:16 +01:00
unknown
008363e87c MDEV-26: Global transaction ID.
Add a missing check for correct table definition.
2013-03-11 16:16:55 +01:00
unknown
492c3b09d2 Fix compile error when building with DBUG, but without DEBUG_SYNC. 2013-03-01 14:58:19 +01:00
Sergei Golubchik
8161c6772d merge with mysql-5.5.30 minus few incorrect or not applicable changesets 2013-02-28 18:42:49 +01:00
unknown
06ffea8c5b MDEV-26: Global Transaction ID
Fix that CHANGE MASTER ... MASTER_GTID_POS="" works to start from the very
beginning of the binary log (with test case).

Fix that not finding the requested GTID position in master binlog results in
fatal error, not endless connect retry.
2013-02-27 21:10:40 +01:00
unknown
861830f9c2 MDEV-26: Global transaction ID.
Remove the two-component form of GTID with implicit domain_id=0, as it
is likely to cause more confusion than help.

Give a better error for CHANGE MASTER ... MASTER_GTID_POS='gtid,gitd,...'
when two specified GTIDs have conflicting domain_id.
2013-02-26 17:25:07 +01:00
unknown
2cf3d61fc2 MDEV-26: Global Transaction ID
- Fix skipping initial MyISAM DML when connecting using GTID.

 - Fix RESET MASTER not clearing in-memory binlog state.

 - Fix not reading standalone flag in Gtid_log_event::peek().

 - Fix skipping DDL that the slave has already seen when using GTID position.
2013-02-22 12:31:55 +01:00
unknown
fff47f40f6 MDEV-26: Global Transaction ID
- Fix that binlog_gtid_pos() (and hence slave connect) does not work
   correctly in the very first binlog file (due to not logging empty
   Gtid_list_log_event).

 - Remove one instance of the stupid domain_id-0-is-implicit.

 - Rename the confusing Gtid_Pos_Auto in SHOW SLAVE STATUS to Using_Gtid.

 - Fix memory leak.
2013-02-22 10:23:17 +01:00
unknown
cb46c33386 MDEV-26: Global transaction ID.
Fix crash in binlog_gtid_pos('', 4) (and thus slave connect) when master does not
have binlog enabled.
2013-02-21 14:46:11 +01:00
unknown
7426a466dc MDEV-26: Global transaction ID, intermediate commit.
Fix memory leak.
2013-02-19 14:22:29 +01:00
unknown
bfec9c64b8 MDEV-26: Global transaction ID, intermediate commit.
- Fix that slave GTID state was updated from the wrong place in the code,
   causing random crashing and other misery.

 - Fix updates to mysql.rpl_slave_state to not go to binlog (this would cause
   duplicate key errors on the slave and is generally the wrong thing to do).
2013-02-19 11:45:29 +01:00
unknown
ff8676e0e1 MDEV-26: Global transaction ID.
Fix initial loading of mysql.rpl_slave_state, the code had several
issues.

Some very basic MTR stuff, more to come.
2013-02-18 15:41:17 +01:00
Pedro Gomes
91e043c304 BUG#13545447: RPL_ROTATE_LOGS FAILS DUE TO CONCURRENCY ISSUES IN REP. CODE
Merge from mysql 5.1 to mysql 5.5
2013-02-15 22:18:37 +00:00
Pedro Gomes
7e8c887786 BUG#13545447: RPL_ROTATE_LOGS FAILS DUE TO CONCURRENCY ISSUES IN REP. CODE
In method mysql_binlog_send, right after detecting a EOF in the
read event loop, and before deciding if we should change to a new
binlog file there is a execution window where new events can be
written to the binlog and a rotation can happen. When reaching
the test, the function will then change to a new binlog file
ignoring all the events written in this window. This will result
in events not being replicated.

Only when the binlog is detected as deactivated in the event loop
of the dump thread, can we really know that no more events
remain. For this reason, this test is now made under the log lock
in the beginning of the event loop when reading the events.
2013-02-15 21:57:35 +00:00
unknown
1c6271b36a MDEV-26: Global Transaction ID.
Implement CHANGE MASTER TO ... MASTER_GTID_POS = "x-y-z,a-b-c".
2013-02-15 15:55:17 +01:00
unknown
d98e2ea9fe MDEV-26. Intermediary commit.
Fix binlog_gtid_pos() to handle empty file name, and to not allow user
to open arbitrary file on the system.
2013-02-13 13:36:46 +01:00
unknown
041f26d349 MDEV-26. Intermediate commit.
Implement binlog_gtid_pos() function. This will be used so that
the slave can obtain the gtid position automatically from first
connect with old-style position - then MASTER_GTID_POS=AUTO will
work the next time. Can also be used by mysqldump --master-data
to give the current gtid position directly.
2013-02-11 16:44:38 +01:00
Sergei Golubchik
ab83952f29 10.0-base merge 2013-01-31 09:48:19 +01:00
unknown
ea5632e9d7 MDEV-3984: Double free of Master_info * when CHANGE MASTER fails.
When CHANGE MASTER fails, it may or may not have already added
the Master_info * to the index. Implement logic that properly
handles removal and freeing in both cases.
2013-01-30 15:11:36 +01:00
Sergei Golubchik
0af4b6c6ee 5.5 merge 2013-01-29 15:10:47 +01:00
Venkatesh Duggirala
f31611c4c5 BUG#14798572: REMOVE UNUSED VARIABLE BINLOG_CAN_BE_CORRUPTED
FROM MYSQL_BINLOG_SEND

As part Bug #11747416 A DISK FULL MAKES BINARY LOG CORRUPT,
reading the variable "binlog_can_be_corrupted" was removed
In the existing code the value of this variable is only set,
never read. And also this issue causing compiler warnings.
So the variable is completely redundant and should be removed.

sql/sql_repl.cc:
  Removing dead code
2013-01-24 10:35:07 +05:30
unknown
0b36233acd MDEV-26. Intermediate commit.
Implement CHANGE MASTER TO MASTER_GTID_POS=xxx.
2013-01-22 15:18:36 +01:00
Michael Widenius
a42e1e3885 Fixed MDEV-4013: Password length in replication setup
Give error for wrong parameters to CHANGE MASTER
Extend MASTER_PASSWORD and MASTER_HOST lengths


mysql-test/suite/rpl/r/rpl_password_boundaries.result:
  Test length of MASTER_PASSWORD, MASTER_HOST and MASTER_USER
mysql-test/suite/rpl/r/rpl_semi_sync.result:
  Use different password than user name for better test coverage
mysql-test/suite/rpl/t/rpl_password_boundaries.test:
  Test length of MASTER_PASSWORD, MASTER_HOST and MASTER_USER
mysql-test/suite/rpl/t/rpl_semi_sync.test:
  Use different password than user name for better test coverage
sql/rpl_mi.h:
  Extend MASTER_PASSWORD and MASTER_HOST lengths
sql/sql_repl.cc:
  Give error for wrong parameters to CHANGE MASTER
sql/sql_repl.h:
  Extend MASTER_PASSWORD and MASTER_HOST lengths
2013-01-11 00:35:33 +02:00
Shivji Kumar Jha
195e731edb BUG#12359942 - REPLICATION TEST FROM ENGINE SUITE RPL_ROW_UNTIL TIMES OUT
bzr merge 5.1->5.5
2012-12-05 10:24:45 +05:30
Shivji Kumar Jha
07ffa9c767 BUG#12359942 - REPLICATION TEST FROM ENGINE SUITE RPL_ROW_UNTIL TIMES OUT
=== Problem ===

The test is dependent on binlog positions and checks
to see if the command 'START SLAVE' functions correctly
with the 'UNTIL' clause added to it. The 'UNTIL' clause
is added to specify that the slave should start and run
until the SQL thread reaches a given point in the master
binary log or in the slave relay log.

The test uses hard coded values for MASTER_LOG_POS and
RELAY_LOG_POS, instead of extracting it using
query_get_value() function. There is a test
'rpl.rpl_row_until' which does the similar thing but uses 
query_get_value() function to set the values of
MASTER_LOG_POS/ RELAY_LOG_POS. To be precise,
rpl.rpl_row_until is a modified version of
engines/func.rpl_row_until.test.

The use of hard coded values may lead the slave to stop at a position
which may differ from the expected position in the binlog file,
an example being the failure of engines/funcs.rpl_row_until in 
mysql-5.1 given as: 
"query 'select * from t2' failed. Table 'test.t2' doesn't exist".
In this case, the slave actually ran a couple of extra commands
as a result of which the slave first deleted the table and then
ran a select query on table, leading to the above mentioned failure.

=== Fix ===

1) Fixed the code for failure seen in rpl.rpl_row_until.
   This test was also failing although the symptoms of
   failure were different.
2) Copied the contents from rpl.rpl_row_until into
   into engines/funcs.rpl.rpl_row_until.
3) Updated engines/funcs.rpl_row_until.result accordingly.

mysql-test/suite/engines/funcs/r/rpl_row_until.result:
  modified to accomodate the changes in corresponding
  test file.
mysql-test/suite/engines/funcs/t/disabled.def:
  removed from the list of disabled tests.
mysql-test/suite/engines/funcs/t/rpl_row_until.test:
  fixed rpl.rpl_row_until and copied its content to
  engines/funcs.rpl_row_until. The reason being both
  are same tests but rpl.rpl_row_until is an
  updated version.
mysql-test/suite/rpl/t/disabled.def:
  removed from the list of disabled tests.
sql/sql_repl.cc:
  Added a check to catch an improper combination
  of arguements passed to 'START SLAVE UNTIL'. Earlier,
  START SLAVE UNTIL MASTER_LOG_FILE='master-bin.000001',
  MASTER_LOG_POS=561, RELAY_LOG_POS=12;
  passed. It is now detected and an error is reported.
2012-11-30 12:12:33 +05:30
unknown
bdbce30dee MDEV-26: Global transaction id: Intermediate commit.
Now slave can connect to master, sending start position as slave state
rather than old-style binlog name/position.

This enables to switch to a new master by changing just connection
information, replication slave GTID state ensures that slave starts
at the correct point in the new master.
2012-11-15 13:11:35 +01:00
unknown
03f28863e8 MDEV-26: Global transaction commit. Intermediate commit.
Now slave records GTID in mysql.rpl_slave_state when applying XID log event.
2012-11-05 15:01:49 +01:00
unknown
ab8e8f4b27 MDEV-500: Session variable for server_id
MDEV-26: Global transaction id, partial commit

Change server_id to be a session variable.
User with SUPER can set it to binlog with different server_id.
Implement backward-compatible ::server_id mirror for plugins.
2012-10-23 12:46:29 +02:00
unknown
be86e44280 MDEV-26: Global transaction id. Partial commit. 2012-10-23 11:19:42 +02:00
Sergei Golubchik
e1f681c99b 10.0-base -> 10.0-monty 2012-10-19 20:38:59 +02:00
Michael Widenius
572560f38c Changed SHOW_FUNC variabels that don't return SHOW_ARRAY to SHOW_SIMPLE_FUNC.
This allows us to avoid calculating variables (including those involving mutex) that doesn't match the given
wildcard in SHOW STATUS LIKE '...'
Removed all references to active_mi that could cause problems for multi-source replication.

Added START|STOP ALL SLAVES
Added SHOW ALL SLAVES STATUS


include/mysql/plugin.h:
  Added SHOW_SIMPLE_FUNC
include/mysql/plugin_audit.h.pp:
  Updated .pp file
include/mysql/plugin_auth.h.pp:
  Updated .pp file
include/mysql/plugin_ftparser.h.pp:
  Updated .pp file
mysql-test/suite/multi_source/info_logs.result:
  New columns in SHOW ALL SLAVES STATUS
mysql-test/suite/multi_source/info_logs.test:
  Test new syntax
mysql-test/suite/multi_source/simple.result:
  New columns in SHOW ALL SLAVES STATUS
mysql-test/suite/multi_source/simple.test:
  test new syntax
mysql-test/suite/multi_source/syntax.result:
  Updated result
mysql-test/suite/multi_source/syntax.test:
  Test new syntax
mysql-test/suite/rpl/r/rpl_skip_replication.result:
  Updated result
plugin/semisync/semisync_master_plugin.cc:
  SHOW_FUNC -> SHOW_SIMPLE_FUNC
sql/item_create.cc:
  Simplify code
sql/lex.h:
  Added SLAVES keyword
sql/log.cc:
  Constant -> define
sql/log_event.cc:
  Added comment
sql/mysqld.cc:
  SHOW_FUNC -> SHOW_SIMPLE_FUNC
  Made slave_retried_trans, slave_received_heartbeats and heartbeat_period multi-source safe
  Clear variable denied_connections and slave_retried_transactions on startup
sql/mysqld.h:
  Added slave_retried_transactions
sql/rpl_mi.cc:
  create_signed_file_name -> create_logfile_name_with_suffix
  Added start_all_slaves() and stop_all_slaves()
sql/rpl_mi.h:
  Added prototypes
sql/rpl_rli.cc:
  create_signed_file_name -> create_logfile_name_with_suffix
  added executed_entries
sql/rpl_rli.h:
  Added executed_entries
sql/share/errmsg-utf8.txt:
  More and better error messages
sql/slave.cc:
  Added more fields to SHOW ALL SLAVES STATUS
  Added slave_retried_transactions
  Changed constants -> defines
sql/sql_class.h:
  Added comment
sql/sql_insert.cc:
  active_mi.rli -> thd->rli_slave
sql/sql_lex.h:
  Added SQLCOM_SLAVE_ALL_START & SQLCOM_SLAVE_ALL_STOP
sql/sql_load.cc:
  active_mi.rli -> thd->rli_slave
sql/sql_parse.cc:
  Added START|STOP ALL SLAVES
sql/sql_prepare.cc:
  Added SQLCOM_SLAVE_ALL_START & SQLCOM_SLAVE_ALL_STOP
sql/sql_reload.cc:
  Made REFRESH RELAY LOG multi-source safe
sql/sql_repl.cc:
  create_signed_file_name -> create_logfile_name_with_suffix
  Don't send my_ok() from start_slave() or stop_slave() so that we can call it for all master connections
sql/sql_show.cc:
  Compare wild cards early for all variables
sql/sql_yacc.yy:
  Added START|STOP ALL SLAVES
  Added SHOW ALL SLAVES STATUS
sql/sys_vars.cc:
  Made replicate_events_marked_for_skip,slave_net_timeout and rpl_filter multi-source safe.
sql/sys_vars.h:
  Simplify Sys_var_rpl_filter
2012-10-03 01:44:54 +03:00
Michael Widenius
8ac1b41cf3 Made max_relay_log_size depending on master connection.
Changed names of multi-source log files so that original suffixes are kept.


include/my_sys.h:
  Added fn_ext2(), which returns pointer to last '.' in file name
mysql-test/extra/rpl_tests/rpl_max_relay_size.test:
  Updated test
mysql-test/suite/multi_source/info_logs-master.opt:
  Test with strange file names
mysql-test/suite/multi_source/info_logs.result:
  Updated results
mysql-test/suite/multi_source/info_logs.test:
  Changed to test with complex names to be able to verify the filename generator code
mysql-test/suite/multi_source/relaylog_events.result:
  Updated results
mysql-test/suite/multi_source/reset_slave.result:
  Updated results
mysql-test/suite/multi_source/skip_counter.result:
  Updated results
mysql-test/suite/multi_source/skip_counter.test:
  Added testing of max_relay_log_size
mysql-test/suite/rpl/r/rpl_row_max_relay_size.result:
  Updated results
mysql-test/suite/rpl/r/rpl_stm_max_relay_size.result:
  Updated results
mysql-test/suite/sys_vars/r/max_relay_log_size_basic.result:
  Updated results
mysql-test/suite/sys_vars/t/max_relay_log_size_basic.test:
  Updated results
mysys/mf_fn_ext.c:
  Added fn_ext2(), which returns pointer to last '.' in file name
sql/log.cc:
  Removed some wrong casts
sql/log.h:
  Updated comment to reflect new code
sql/log_event.cc:
  Updated DBUG_PRINT
sql/mysqld.cc:
  Added that max_relay_log_size copies it's values from max_binlog_size
sql/mysqld.h:
  Removed max_relay_log_size
sql/rpl_mi.cc:
  Changed names of multi-source log files so that original suffixes are kept.
sql/rpl_mi.h:
  Updated prototype
sql/rpl_rli.cc:
  Updated comment to reflect new code
  Made max_relay_log_size depending on master connection.
sql/rpl_rli.h:
  Made max_relay_log_size depending on master connection.
sql/set_var.h:
  Made option global so that one can check and change min & max values (sorry Sergei)
sql/sql_class.h:
  Made max_relay_log_size depending on master connection.
sql/sql_repl.cc:
  Updated calls to create_signed_file_name()
sql/sys_vars.cc:
  Made max_relay_log_size depending on master connection.
  Made old code more reusable
sql/sys_vars.h:
  Changed Sys_var_multi_source_uint to ulong to be able to handle max_relay_log_size
  Made old code more reusable
2012-10-01 02:30:44 +03:00
Michael Widenius
5a4b5869a0 Fixed issues found by QA (Elena)
- Added parameter to reset_logs() so that one can specify if new logs should be created.

mysql-test/include/setup_fake_relay_log.inc:
  There is no orphan relay log files anymore
mysql-test/mysql-test-run.pl:
  Added multi_source to test suite
mysql-test/suite/multi_source/info_logs.result:
  New test
mysql-test/suite/multi_source/info_logs.test:
  New test
mysql-test/suite/multi_source/my.cnf:
  Added log-warnings to get more information to the log files
mysql-test/suite/multi_source/relaylog_events.result:
  Added cleanup
mysql-test/suite/multi_source/relaylog_events.test:
  Added cleanup
mysql-test/suite/multi_source/reset_slave.result:
  Updated results after improved RESET SLAVE
mysql-test/suite/multi_source/simple.result:
  Updated results after improved RESET SLAVE
mysql-test/suite/multi_source/simple.test:
  Syncronize positions before show full slave status
mysql-test/suite/rpl/r/rpl_row_show_relaylog_events.result:
  Updated results after improved RESET SLAVE (we now use less relay log files)
mysql-test/suite/rpl/r/rpl_stm_mix_show_relaylog_events.result:
  Updated results after improved RESET SLAVE (we now use less relay log files)
sql/log.cc:
  Added parameter to reset_logs() so that one can specify if new logs should be created.
sql/log.h:
  Added parameter to reset_logs()
sql/rpl_mi.cc:
  Create Master_info_index::index_file_names once at init
  More DBUG_PRINT
  Give error if Master_info_index::check_duplicate_master_info fails
sql/rpl_rli.cc:
  If we do a full reset, don't create any new relay log files.
sql/share/errmsg-utf8.txt:
  Improved error message if connection exists
sql/sql_parse.cc:
  Fixed memory leak
sql/sql_repl.cc:
  check_duplicate_master_info() now generates an error
  Added parameter to reset_logs()
2012-09-28 21:22:24 +03:00
Michael Widenius
222883b0d6 Added multi-source support to show relaylog events 2012-09-28 03:45:05 +03:00