Commit Graph

2686 Commits

Author SHA1 Message Date
Marko Mäkelä
a5d3c1c819 Merge 10.4 into 10.5 2021-03-08 10:16:20 +02:00
Marko Mäkelä
39e2c95771 Merge 10.3 into 10.4 2021-03-08 09:09:31 +02:00
Igor Babaev
08d8bce583 MDEV-22786 Crashes with nested table value constructors
The bug caused crashes of the server when processing queries with nested
table value constructors (TVC) . It happened because the grammar rules to
parse TVC used the same global lists for both nested TVC and nesting TVC.
As a result invalid select trees were constructed for queries with nested
TVC and this led to crashes at the prepare stage.
This patch provides its own lists structures for each TVC nest level.

Besides the patch fixes a bug in the function wrap_tvc() that missed
inheritance of the SELECT_LEX::exclude_from_table_unique_test for
selects that wrapped TVCs. This inheritance is critical for specifications
of derived tables that employ nested TVCs.

Approved by dmitry.shulga@mariadb.com
2021-03-03 22:49:29 -08:00
Daniel Black
86d60fc9e7 Merge remote-tracking branch 'origin/10.4' into 10.5 2021-02-26 13:23:13 +11:00
Daniel Black
36810342d5 Merge branch '10.3' into 10.4 2021-02-25 13:16:10 +11:00
Igor Babaev
bf6484e7bb MDEV-24910 Crash with SELECT that uses table value constructor as a subselect
This bug caused crashes of the server when processing queries with table
value constructors (TVC) that contained subqueries and were used itself as
subselects. For such TVCs the following transformation is applied at the
prepare stage:
  VALUES (v1), ... (vn) => SELECT * FROM (VALUES (v1), ... (vn)) tvc_x.
This transformation allows to reduce the problem of evaluation of TVCs used
as subselects to the problem of evaluation of regular subselects.
The transformation is implemented in the wrap_tvc(). The code the function
to mimic the behaviour of the parser when processing the result of the
transformation. However this imitation was not free of some flaws. First
the function called the method exclude() that completely destroyed the
select tree structures below the transformed TVC. Second the function
used the procedure mysql_new_select to create st_select_lex nodes for
both wrapping select of the transformation and TVC. This also led to
constructing of invalid select tree structures.
The patch actually re-engineers the code of wrap_tvc().

Approved by Oleksandr Byelkin <sanja@mariadb.com>
2021-02-24 13:51:47 -08:00
Sergei Golubchik
f33e57a9e6 Merge branch '10.4' into 10.5 2021-02-23 13:06:22 +01:00
Sergei Golubchik
e841957416 Merge branch '10.3' into 10.4 2021-02-23 09:25:57 +01:00
Sergei Golubchik
0ab1e3914c Merge branch '10.2' into 10.3 2021-02-22 22:42:27 +01:00
Sujatha
eef4c5d378 MDEV-22741: *SAN: ERROR: AddressSanitizer: use-after-poison on address in instrings/strmake.c:36 from change_master (on optimized builds)
Problem:
========
CHANGE MASTER TO MASTER_USER='root', MASTER_SSL=0, MASTER_SSL_CA='',
  MASTER_SSL_CERT='', MASTER_SSL_KEY='', MASTER_SSL_CRL='',
  MASTER_SSL_CRLPATH='';

CHANGE MASTER TO MASTER_USER='root', MASTER_PASSWORD='', MASTER_SSL=0;

use-after-poison is reported for lex_mi->ssl_crl

File: sql_repl.cc

if (lex_mi->ssl_crl)
  strmake_buf(mi->ssl_crl, lex_mi->ssl_crl);

Analysis:
========
At the end of CHANGE MASTER statement execution, the LEX_MASTER_INFO
parameters are reset so that the next query will have a clean state. But
'ssl_crl' and 'ssl_crl_path' members of LEX_MASTER_INFO object are not
cleared during 'LEX_MASTER_INFO::reset'. Hence when a new CHANGE MASTER
statement is executed, the stale value of lex_mi->ssl_crl is used, so ASAN
reports use-after-poison.

Fix:
===
Clear 'ssl_crl' and 'ssl_crl_path' as part of 'reset'.
2021-02-03 12:18:29 +05:30
Monty
c8992fc35b Trivial cleanups, no logic changes
- Fold long comment rows and updated comments
- Moved one private function in class Item_func_rand among other private
  functions
2020-11-26 19:13:37 +02:00
Marko Mäkelä
d7a5824899 Merge 10.4 into 10.5 2020-11-13 21:54:21 +02:00
Nayuta Yanagisawa
b13fe8e51b MDEV-18842: Unfortunate error message when the same column is used for application period start and end
An application-time period must be composed of two different columns.
We added a check that ensures that the above condition is met.
2020-11-04 12:33:13 +03:00
Alexander Barkov
080522dcd7 MDEV-23825 Join select_handler and Pushdown_select + XPand changes 2020-09-27 10:30:23 +04:00
Marko Mäkelä
5ff7e68c7e Merge 10.4 into 10.5 2020-09-04 18:44:44 +03:00
Oleksandr Byelkin
0f080dd60a MDEV-23094: Multiple calls to a Stored Procedure from another Stored Procedure crashes server
Added system-SELECT to IF/WHILE/REPET/FOR for correct subqueries connecting.

Added control of system/usual selects for correct error detection.
2020-08-31 14:40:34 +02:00
Oleksandr Byelkin
48b5777ebd Merge branch '10.4' into 10.5 2020-08-04 17:24:15 +02:00
Oleksandr Byelkin
57325e4706 Merge branch '10.3' into 10.4 2020-08-03 14:44:06 +02:00
Marko Mäkelä
50a11f396a Merge 10.4 into 10.5 2020-08-01 14:42:51 +03:00
Alexander Barkov
d63631c3fa MDEV-19632 Replication aborts with ER_SLAVE_CONVERSION_FAILED upon CREATE ... SELECT in ORACLE mode
- Adding optional qualifiers to data types:
    CREATE TABLE t1 (a schema.DATE);
  Qualifiers now work only for three pre-defined schemas:

    mariadb_schema
    oracle_schema
    maxdb_schema

  These schemas are virtual (hard-coded) for now, but may turn into real
  databases on disk in the future.

- mariadb_schema.TYPE now always resolves to a true MariaDB data
  type TYPE without sql_mode specific translations.

- oracle_schema.DATE translates to MariaDB DATETIME.

- maxdb_schema.TIMESTAMP translates to MariaDB DATETIME.

- Fixing SHOW CREATE TABLE to use a qualifier for a data type TYPE
  if the current sql_mode translates TYPE to something else.

The above changes fix the reported problem, so this script:

    SET sql_mode=ORACLE;
    CREATE TABLE t2 AS SELECT mariadb_date_column FROM t1;

is now replicated as:

    SET sql_mode=ORACLE;
    CREATE TABLE t2 (mariadb_date_column mariadb_schema.DATE);

and the slave can unambiguously treat DATE as the true MariaDB DATE
without ORACLE specific translation to DATETIME.

Similar,

    SET sql_mode=MAXDB;
    CREATE TABLE t2 AS SELECT mariadb_timestamp_column FROM t1;

is now replicated as:

    SET sql_mode=MAXDB;
    CREATE TABLE t2 (mariadb_timestamp_column mariadb_schema.TIMESTAMP);

so the slave treats TIMESTAMP as the true MariaDB TIMESTAMP
without MAXDB specific translation to DATETIME.
2020-08-01 07:43:50 +04:00
Oleksandr Byelkin
e08f87d527 fix obvious bugs hidden by current_select assigned to builtin select 2020-07-28 08:23:57 +02:00
Oleksandr Byelkin
c6eb21cd87 MDEV-21998: Server crashes in st_select_lex::add_table_to_list upon mix of KILL and sequences
Continue support the hack of current select equal builtin select if
selects stack is empty even after subselects.
2020-07-28 08:23:57 +02:00
Marko Mäkelä
e67daa5653 Merge 10.4 into 10.5 2020-07-15 14:51:22 +03:00
Marko Mäkelä
9936cfd531 Merge 10.3 into 10.4 2020-07-15 10:17:15 +03:00
Marko Mäkelä
8a0944080c Merge 10.2 into 10.3 2020-07-14 22:59:19 +03:00
Varun Gupta
b0df247db6 MDEV-22463: Element_type &Bounds_checked_array<Item *>::operator[](size_t) [Element_type = Item *]: Assertion `n < m_size' failed.
Allocate space for fields inside the window function (arguments, PARTITION BY and ORDER BY clause)
in the ref pointer array. All fields inside the window function are part of the temporary
table that is required for the window function computation.
2020-07-13 22:04:54 +05:30
Alexander Barkov
5967dfdbbf MDEV-23154 Add a data type my_repertoire_t 2020-07-13 19:50:07 +04:00
Marko Mäkelä
1813d92d0c Merge 10.4 into 10.5 2020-07-02 09:41:44 +03:00
Aleksey Midenkov
762bf7a03b MDEV-22602 Disable UPDATE CASCADE for SQL constraints
CHECK constraint is checked by check_expression() which walks its
items and gets into Item_field::check_vcol_func_processor() to check
for conformity with foreign key list.

WITHOUT OVERLAPS is checked for same conformity in
mysql_prepare_create_table().

Long uniques are already impossible with InnoDB foreign keys. See
ER_CANT_CREATE_TABLE in test case.

2 accompanying bugs fixed (test main.constraints failed):

1. check->name.str lived on SP execute mem_root while "check" obj
itself lives on SP main mem_root. On second SP execute check->name.str
had garbage data. Fixed by allocating from thd->stmt_arena->mem_root
which is SP main mem_root.

2. CHECK_CONSTRAINT_IF_NOT_EXISTS value was mixed with
VCOL_FIELD_REF. VCOL_FIELD_REF is assigned in check_expression() and
then detected as CHECK_CONSTRAINT_IF_NOT_EXISTS in
handle_if_exists_options().

Existing cases for MDEV-16932 in main.constraints cover both fixes.
2020-06-12 11:12:40 +03:00
Alexander Barkov
0bf843cd13 MDEV-20366 Server crashes in get_current_user upon SET PASSWORD via SP
The opt_for_user subrule was incorrectly scanned before sp_create_assignment_lex(),
so the user name and the host were created on a wrong memory root.

- Reoganizing the grammar to make sure that sp_create_assignment_lex()
  is called immediately after PASSWORD_SYM is scanned, so all attributes
  are then allocated on its memory root.

- Moving the semantic code as methods to LEX, so the grammar looks as simple as possible.

- Changing text_or_password to be of the data type USER_AUTH*.
  As a side effect, the LEX::definer member is now not used when processing
  the SET PASSWORD statement. Everything is done using Bison's stack.

The bug sas introduced by this commit:
commit bf5a144e16
2020-05-30 14:00:56 +04:00
Aleksey Midenkov
431a740815 MDEV-21889 IF EXISTS clause does not work for RENAME COLUMN and RENAME INDEX 2020-04-04 00:53:36 +03:00
Monty
6a9e24d046 Added support for replication for S3
MDEV-19964 S3 replication support

Added new configure options:
s3_slave_ignore_updates
"If the slave has shares same S3 storage as the master"

s3_replicate_alter_as_create_select
"When converting S3 table to local table, log all rows in binary log"

This allows on to configure slaves to have the S3 storage shared or
independent from the master.

Other thing:
Added new session variable '@@sql_if_exists' to force IF_EXIST to DDL's.
2020-03-24 21:00:02 +02:00
Sergei Golubchik
c1c5222cae cleanup: PSI key is *always* the first argument 2020-03-10 19:24:23 +01:00
Sergei Golubchik
70e7b5095d perfschema sp instrumentation related changes 2020-03-10 19:24:23 +01:00
Sergei Golubchik
7c58e97bf6 perfschema memory related instrumentation changes 2020-03-10 19:24:22 +01:00
Sergei Petrunia
0a9633ee62 Basic LEX::print function that supports UPDATEs 2020-03-07 01:26:28 +03:00
Aleksey Midenkov
fa8ad75439 MDEV-16290 ALTER TABLE ... RENAME COLUMN syntax
The existing syntax for renaming a column uses "ALTER TABLE ...  CHANGE"
command. This requires full column specification to rename the column.
This patch adds new syntax "ALTER TABLE ...  RENAME COLUMN", which do not
expect users to provide full column specification.  It means that the new
syntax would pick in-place or copy algorithm in the same way as that of
existing "ALTER TABLE ... CHANGE" command. The existing syntax
"ALTER TABLE ... CHANGE" will continue to work.

Syntax changes
==============

ALTER TABLE tbl_name
    [alter_specification [, alter_specification] ...]
    [partition_options]

Following is a new <alter_specification> added:

 | RENAME COLUMN <oldname> TO <newname>

Where <oldname> and <newname> are identifiers for old name and new
name of the column.

Related to: WL#10761
2020-03-03 13:50:32 +03:00
Alexander Barkov
83e75b39b3 MDEV-21702 Add a data type for privileges 2020-02-11 08:10:26 +04:00
Alexander Barkov
77c6382312 MDEV-21689 Add Sql_cmd for GRANT/REVOKE statements
Rewriting GRANT/REVOKE grammar to use more bison stack and use Sql_cmd_ style

1. Removing a few members from LEX:
   - uint grant, grant_to_col, which_columns
   - List<LEX_COLUMN> columns
   - bool all_privileges
2. Adding classes Grand_object_name, Lex_grant_object_name
3. Adding classes Grand_privilege, Lex_grand_privilege
4. Adding struct Lex_column_list_privilege_st, class Lex_column_list_privilege
5. Rewriting the GRANT/REVOKE grammar to use new classes and pass them through
   bison stack (rather than directly access LEX members)
6. Adding classes Sql_cmd_grant* and Sql_cmd_revoke*,
   changing GRANT/REVOKE to use LEX::m_sql_cmd.
7. Adding the "sp_handler" grammar rule and removing some duplicate grammar
   for GRANT/REVOKE for different kinds of SP objects.
8. Adding a new rule comma_separated_ident_list, reusing it in:
   - with_column_list
   - colum_list_privilege
2020-02-08 21:35:35 +04:00
Marko Mäkelä
8cc15c036d Merge 10.4 into 10.5 2019-12-27 21:17:16 +02:00
Marko Mäkelä
4c25e75ce7 Merge 10.3 into 10.4 2019-12-27 18:20:28 +02:00
Marko Mäkelä
5ab70e7f68 Merge 10.2 into 10.3 2019-12-27 15:14:48 +02:00
Oleksandr Byelkin
cc28947315 MDEV-20632: prerequisite:
Removed hack with with_list
2019-12-27 10:49:52 +01:00
Marko Mäkelä
73985d8301 Merge 10.1 into 10.2 2019-12-23 07:14:51 +02:00
Sergei Petrunia
8b9db11718 MDEV-21341: Fix optimizer-related UBSAN failures, part #2
Remove Query_tables_list::lock_tables_state - it is not used and it causes
errors like this:

sql_lex.h:1675:7: runtime error: load of value 2779096485, which is not a
valid value for type 'enum_lock_tables_state'
2019-12-18 12:56:54 +03:00
Aleksey Midenkov
0c05a2ed71 Merge 10.4 into 10.5 2019-11-25 17:24:09 +03:00
Aleksey Midenkov
33f55789d3 MDEV-18727 improve DML operation of System Versioning (10.4)
UPDATE, DELETE: replace linear search of current/historical records
with vers_setup_conds().

Additional DML cases in view.test
2019-11-25 16:01:43 +03:00
Aleksey Midenkov
daabc5cc68 MDEV-20729 Fix REFERENCES constraint in column definition
Add support of referential constraints directly in column defininions:

create table t1 (id1 int primary key);
create table t2 (id2 int references t1(id1));

Referenced field name can be omitted if equal to foreign field name:

create table t1 (id int primary key);
create table t2 (id int references t1);

Until 10.5 this syntax was understood by the parser but was silently
ignored.

In case of generated columns this syntax is disabled at parser level
by ER_PARSE_ERROR. Note that separate FOREIGN KEY clause for generated
columns is disabled at storage engine level.
2019-11-20 13:18:54 +03:00
Alexander Barkov
251c6e1726 MDEV-21073 Collect different grammar rules into a single chunk
On order to unify the two *.yy files easier,
this patch collects all different rules to the end of *.yy files,
so the rule section looks like this:

%%
common rules
different rules
2019-11-19 08:06:04 +04:00
Alexander Barkov
6d373e8b81 MDEV-21064 Add a new class sp_expr_lex and a new grammar rule expr_lex
Adding:
- new class sp_expr_lex
- new grammar rule expr_lex, which includes both reset_lex()
  and its corresponding restore_lex()

Also:
- Moving a few methods from LEX to sp_expr_lex.
- Moving the code from *.yy to new method sp_expr_lex methods
  sp_repeat_loop_finalize() and sp_if_expr().

This change makes it easier to edit the related grammar
(and makes it easier to unify sql_yacc.yy and sql_yacc_ora.yy later).
2019-11-16 08:32:15 +04:00
Oleksandr Byelkin
3ad37ed0eb Merge 10.4 into 10.5 2019-11-07 08:52:30 +01:00
Alexander Barkov
46f2f24ec4 MDEV-20985 Add LEX methods stmt_drop_{function|procedure}() and stmt_alter_{function|procedure}_start()
Adding a few helper LEX methods,
to unify sql_yacc.yy and sql_yacc_ora.yy easier
2019-11-05 22:37:45 +04:00
Marko Mäkelä
ec40980ddd Merge 10.3 into 10.4 2019-11-01 15:23:18 +02:00
Alexander Barkov
f2cff12556 MDEV-20924 Unify grammar rules: field_type_string and sp_param_field_type_string
In order to:
- unify sql_yacc.yy and sql_yacc_ora.yy easier
- move more functionality from the parser to Type_handler
  (so plugins can override the behavior)

this patch:
- removes rules sp_param_field_type_string and sp_param_field_type
  from sql_yacc_ora.yy
- adds a new virtial method Type_handler::Column_definition_set_attributes()
2019-10-30 11:58:52 +04:00
Michael Widenius
716d396bb3 Remove \n from DBUG_PRINT statements 2019-10-21 18:41:58 +03:00
Monty
b62101f84b Fixes for binary logging --read-only mode
- Any temporary tables created under read-only mode will never be logged
  to binary log.  Any usage of these tables to update normal tables, even
  after read-only has been disabled, will use row base logging (as the
  temporary table will not be on the slave).
- Analyze, check and repair table will not be logged in read-only mode.

Other things:
- Removed not used varaibles in
  MYSQL_BIN_LOG::flush_and_set_pending_rows_event.
- Set table_share->table_creation_was_logged for all normal tables.
- THD::binlog_query() now returns -1 if statement was not logged., This
  is used to update table_share->table_creation_was_logged.
- Don't log admin statements in opt_readonly is set.
- Table's that doesn't have table_creation_was_logged will set binlog format to row
  logging.
- Removed not needed/wrong setting of table->s->table_creation_was_logged
  in create_table_from_items()
2019-10-20 11:52:29 +03:00
Oleksandr Byelkin
de2186dd2f MDEV-20074: Lost connection on update trigger
Instead of checking lex->sql_command which does not corect in case of triggers
mark tables for insert.
2019-10-17 17:32:14 +02:00
Rucha Deodhar
837ad9ab97 MDEV-10014 Add RETURNING to INSERT
Closes #1384
2019-10-14 10:29:31 +02:00
Sergei Golubchik
57a09a72a3 cleanup st_select_lex_unit::explainable 2019-10-14 10:29:31 +02:00
Sergei Golubchik
721a9df751 cleanup: formatting
comments, whitespaces
2019-10-14 10:29:31 +02:00
Oleksandr Byelkin
1ae02f0e0d MDEV-18553: MDEV-16327 pre-requisits part 2: uniform of LIMIT/OFFSET handling
Now both offset and limit are stored and do not chenged during execution
(offset is decreased during processing in versions before 10.5).

(Big part of this changes made by Monty)
2019-10-13 09:40:41 +02:00
Oleksandr Byelkin
eb0804ef5e MDEV-18553: MDEV-16327 pre-requisits part 1: isolation of LIMIT/OFFSET handling 2019-10-13 09:40:41 +02:00
Alexander Barkov
d168601e83 MDEV-20734 Allow reserved keywords as user defined type names 2019-10-03 14:02:00 +04:00
Alexander Barkov
edef6a0074 Merge remote-tracking branch 'origin/10.4' into 10.5 2019-09-24 12:53:11 +04:00
Marko Mäkelä
1333da90b5 Merge 10.4 into 10.5 2019-09-24 10:07:56 +03:00
Igor Babaev
b44171428a MDEV-19956 Queries with subqueries containing UNION are not parsed
Shift-Reduce conflicts prevented parsing some queries with subqueries that
used set operations when the subqueries occurred in expressions or in IN
predicands.
The grammar rules for query expression were transformed in order to avoid
these conflicts. New grammar rules employ an idea taken from MySQL 8.0.
2019-09-23 09:57:37 -07:00
Alexander Barkov
2f88bd2da2 MDEV-20634 Report disallowed subquery errors as such (instead of parse error) 2019-09-20 10:36:20 +04:00
WayneXia
a896bebfa6 MDEV-18844 Implement EXCEPT ALL and INTERSECT ALL operations 2019-08-24 21:44:41 +08:00
Alexander Barkov
c1599821a5 Merge remote-tracking branch 'origin/10.4' into 10.5 2019-08-13 23:49:10 +04:00
Oleksandr Byelkin
2792c6e7b0 Merge branch '10.3' into 10.4 2019-07-28 13:43:26 +02:00
Oleksandr Byelkin
d97342b6f2 Merge branch '10.2' into 10.3 2019-07-26 22:42:35 +02:00
Oleksandr Byelkin
cf8c2a3c3b Merge branch '10.1' into 10.2 2019-07-26 07:03:39 +02:00
Oleksandr Byelkin
ae476868a5 Merge branch '5.5' into 10.1 2019-07-25 13:27:11 +02:00
Gleb Shchepa
7473a71a28 Bug #29419820: MEMORY LEAK IN MY_YYOVERFLOW()
Note: this patch is for 5.6.

Detected by ASAN.

The patch fixes the cleanup of parser stack pointers.

Reviewed-by: Guilhem Bichot <guilhem.bichot@oracle.com>
2019-07-24 18:32:24 +02:00
Igor Babaev
8540fa83bb MDEV-19421 Basic 3-way join queries are not parsed.
The parser returned a syntax error message for the queries with join
expressions like this t1 JOIN t2 [LEFT | RIGHT] JOIN t3 ON ... ON ... when
the second operand of the outer JOIN operation with ON clause was another
join expression with ON clause. In this expression the JOIN operator is
right-associative, i.e. expression has to be parsed as the expression
t1 JOIN (t2 [LEFT | RIGHT] JOIN t3 ON ... ) ON ...
Such join expressions are hard to parse because the outer JOIN is
left-associative if there is no ON clause for the first outer JOIN operator.
The patch implements the solution when the JOIN operator is always parsed
as right-associative and builds first the right-associative tree. If it
happens that there is no corresponding ON clause for this operator the
tree is converted to left-associative.

The idea of the solution was taken from the patch by Martin Hansson
"WL#8083: Fixed the join_table rule" from MySQL-8.0 code line.
As the grammar rules related to join expressions in MySQL-8.0 and
MariaDB-5.5+ are quite different MariaDB solution could not borrow
any code from the MySQL-8.0 solution.
2019-07-11 13:39:21 -07:00
Alexander Barkov
4dc85973b4 MDEV-19994 Add class Function_collection 2019-07-09 16:18:54 +04:00
Alexander Barkov
bf5a144e16 MDEV-19639 + MDEV-19640 fix + preparatory changes for WL#4179
This patch includes:
- MDEV-19639 sql_mode=ORACLE: Wrong SHOW PROCEDURE output for sysvar:=expr
- MDEV-19640 Wrong SHOW PROCEDURE output for SET GLOBAL sysvar1=expr, sysvar2=expr
- Preparatory refactoring for MySQL WL#4179

Detailed change list:

1. Changing sp_create_assignment_lex() to accept the position
   in the exact query buffer instead of a "bool no_lookahead".
   This actually fixes MDEV-19639.
   In the previous reduction sp_create_assignment_lex() was
   called too late, when the parser went far the from beginning
   of the statement, so only a part of the statement got into
   sp_instr_stmt.

2. Generating "SET" or "SET GLOBAL" inside sp_create_assignment_instr()
   depending on the option type.
   This fixes MDEV-19640.
   In the previous reduction the code passed (through no_lookahead)
   the position of the
   word GLOBAL inside sp_create_assignment_lex(), which
   worked only for the left-most assignment.

3. Fixing the affected rules to use:
   - ident_cli instead of ident
   - ident_cli_set_usual_case instead of ident_set_usual_case

4. Changing the input parameter in:
   - LEX::set_system_variable()
   - LEX::call_statement_start()
   - LEX::set_variable()
   from just LEX_CSTRING to Lex_ident_sys_st for stricter data type constrol:
   to make sure that noone passes an ident_cli
   (a fragment of the original query in the client character set)
   instead of server-side identifier
   (utf8 identifier allocated on THD when needed).

5. Adding Lex_ident_sys() in places where the affected functions are called.

6. Moving all calls of sp_create_assignment_lex() to the places
   just before parsing set_expr_or_default.
   This makes the grammar clearer, because
   sp_create_assignment_lex() and sp_create_assignment_instr()
   now stay near each other, so the balance of LEX's push/pop
   can be read easier.
   This will also help to WL#4179.

7. Adding class sp_lex_set_var
   Moving the initialization code from
   sp_create_assignment_lex() to the constructor of sp_lex_set_var.
   This will also help to WL#4179.

8. Moving a part of the "set" grammar rule into a separate
   rule "set_param".
   This makes the grammar easier to read and removes
   one shift/reduce conflict.
2019-06-04 06:21:22 +04:00
Marko Mäkelä
28fad39de7 Merge 10.4 into 10.5 2019-05-29 22:29:05 +03:00
Marko Mäkelä
f98bb23168 Merge 10.3 into 10.4 2019-05-29 22:17:00 +03:00
Marko Mäkelä
90a9193685 Merge 10.2 into 10.3 2019-05-29 11:32:46 +03:00
Alexander Barkov
9f23f8e598 MDEV-19599 Change db_name, table_name to LEX_CSTRING in Item_ident and Send_field 2019-05-26 09:49:13 +04:00
Alexander Barkov
8164bd24a6 MDEV-19535 sql_mode=ORACLE: 'SELECT INTO @var FOR UPDATE' does not lock the table 2019-05-21 14:22:49 +04:00
Alexander Barkov
ed39181a27 MDEV-19533 Add methods make() and append_uniq() to Row_definition_list 2019-05-21 12:30:21 +04:00
Sujatha
5a2110e7cf MDEV-19076: rpl_parallel_temptable result mismatch '-33 optimistic'
Problem:
========
The test now fails with the following trace:

CURRENT_TEST: rpl.rpl_parallel_temptable
--- /mariadb/10.4/mysql-test/suite/rpl/r/rpl_parallel_temptable.result
+++ /mariadb/10.4/mysql-test/suite/rpl/r/rpl_parallel_temptable.reject
@@ -194,7 +194,6 @@
 30    conservative
 31    conservative
 32    optimistic
-33    optimistic

Analysis:
=========
The part of test which fails with result content mismatch is given below.

CREATE TEMPORARY TABLE t4 (a INT PRIMARY KEY) ENGINE=InnoDB;
INSERT INTO t4 VALUES (32);
INSERT INTO t4 VALUES (33);
INSERT INTO t1 SELECT a, "optimistic" FROM t4;

slave_parallel_mode=optimistic

The expectation of the above test script is, INSERT FROM SELECT should read both
32, 33 and populate table 't1'. But this expectation fails occasionally.

All three INSERT statements are handed over to three different slave parallel
workers. Temporary tables are not safe for parallel replication. They were
designed to be visible to one thread only, so have no table locking.  Thus there
is no protection against two conflicting transactions committing in parallel and
things like that.

So anything that uses temporary tables will be serialized with anything before
it, when using parallel replication by using a "wait_for_prior_commit" function
call. This will ensure that the each transaction is executed sequentially.

But there exists a code path in which the above wait doesn't happen.  Because of
this at times INSERT from SELECT doesn't wait for the INSERT (33) to complete
and it completes its executes and enters commit stage.  Hence only row 32 is
found in those cases resulting in test failure.

The wait needs to be added within "open_temporary_table" call. The code looks
like this within "open_temporary_table".

Each thread tries to open temporary table in 3 different ways:

case 1: Find a temporary table which is already in use by using
         find_temporary_table(tl) && wait_for_prior_commit()
case 2: If above failed then try to look for temporary table which is marked for
        free for reuse. This internally calls "wait_for_prior_commit()" if table
        is found.
         find_and_use_tmp_table(tl, &table)
case 3: If none of the above open a new table handle from table share.
         if (!table && (share= find_tmp_table_share(tl)))
         { table= open_temporary_table(share, tl->get_table_name(), true); }

At present the "wait_for_prior_commit" happens only in case 1 & 2.

Fix:
====
On slave add a call for "wait_for_prior_commit" for case 3.

The above wait on slave will solve the issue. A more detailed fix would be to
mark temporary tables as not safe for parallel execution on the master side.
In order to do that, on the master side, mark the Gtid_log_event specific flag
FL_TRANSACTIONAL to be false all the time. So that they are not scheduled
parallely.
2019-05-20 15:46:26 +05:30
Oleksandr Byelkin
dafe41edea Removing of dead code. 2019-05-20 09:38:08 +02:00
Oleksandr Byelkin
c07325f932 Merge branch '10.3' into 10.4 2019-05-19 20:55:37 +02:00
Marko Mäkelä
be85d3e61b Merge 10.2 into 10.3 2019-05-14 17:18:46 +03:00
Marko Mäkelä
26a14ee130 Merge 10.1 into 10.2 2019-05-13 17:54:04 +03:00
Oleksandr Byelkin
c51f85f882 Merge branch '10.2' into 10.3 2019-05-12 17:20:23 +02:00
Vicențiu Ciorbaru
cb248f8806 Merge branch '5.5' into 10.1 2019-05-11 22:19:05 +03:00
Vicențiu Ciorbaru
5543b75550 Update FSF Address
* Update wrong zip-code
2019-05-11 21:29:06 +03:00
Igor Babaev
09aa5d3f69 MDEV-17894 Assertion `(thd->lex)->current_select' failed in MYSQLparse(),
query with VALUES()

A table value constructor can be used in all contexts where a select
can be used. In particular an ORDER BY clause or a LIMIT clause or both
of them can be attached to a table value constructor to produce a new
query. Unfortunately execution of such queries was not supported.
This patch fixes the problem.
2019-05-08 09:45:24 -07:00
Marko Mäkelä
d3dcec5d65 Merge 10.3 into 10.4 2019-05-05 15:06:44 +03:00
Oleksandr Byelkin
8cbb14ef5d Merge branch '10.1' into 10.2 2019-05-04 17:04:55 +02:00
Varun Gupta
cb9fa1a08b MDEV-9959: A serious MariaDB server performance bug
If a derived table has SELECT DISTINCT, provide index statistics for it so that the join optimizer in the
upper select knows that ref access to the table will produce one row.
2019-04-30 21:07:25 +05:30
Sergei Golubchik
2ce52790ff Merge branch '5.5' into 10.1 2019-04-26 14:02:37 +02:00
Sergei Golubchik
5057d46375 bugfix: multi-update checked privileges on views incorrectly
it always required UPDATE privilege on views, not being able to detect
when a views was not actually updated in multi-update.

fix: instead of marking all tables as "updating" by default,
only set "updating" on tables that will actually be updated
by multi-update. And mark the view "updating" if any of the
view's tables is.
2019-04-24 11:15:38 +02:00
Alexander Barkov
c2a2e72164 MDEV-19142 sql_mode=MSSQL: Bracket identifiers 2019-04-15 09:39:28 +04:00
Igor Babaev
d2013e7328 MDEV-18982 Partition pruning with column list causes syntax error in 10.4
A syntax error was reported for any INSERT statement with explicit
partition selection it if i used a column list.
Fixed by saving the parsing place before parsing the clause for explicit
partition selection and restoring it when the clause has been parsed.
2019-04-04 16:36:26 -07:00
Galina Shalygina
ae15f91f22 MDEV-18769 Assertion `fixed == 1' failed in Item_cond_or::val_int
This bug is caused by pushdown from HAVING into WHERE.
    It appears because condition that is pushed wasn't fixed.

    It is also discovered that condition pushdown from HAVING into
    WHERE is done wrong. There is no need to build clones for some
    conditions that can be pushed. They can be simply moved from HAVING
    into WHERE without cloning.
    build_pushable_cond_for_having_pushdown(),
    remove_pushed_top_conjuncts_for_having() methods are changed.

    It is found that there is no transformation made for fields of
    pushed condition.
    field_transformer_for_having_pushdown transformer is added.

    New tests are added. Some comments are changed.
2019-04-04 18:06:56 +03:00
Alexander Barkov
2d825e97df Cleanup: removing unused type LEX_TYPE and #include 2019-04-01 10:09:24 +04:00
Marko Mäkelä
8b480df63e Merge 10.3 into 10.4 2019-03-25 17:18:15 +02:00
Eugene Kosov
53216091dd remove dead code 2019-03-22 22:33:27 +04:00
Eugene Kosov
a523444eda remove dead code 2019-03-22 22:30:36 +04:00
Sergey Vojtovich
3568427d11 MDEV-18450 Slaves wait shutdown
The patches features an optional shutdown behavior to hold on until
after all connected slaves have been sent the last binlogged event.
The connected slave is one whose START SLAVE has been acknowledged and
that was not stopped since that though it could be technically
reconnecting in background.

The solution therefore disallows killing the dump thread until is has
found EOF of the latest binlog file.  It is up to the shutdown
requester (DBA) to set up a sufficiently large shutdown timeout value
for shudown to wait patiently until lagging behind slaves have been
synchronized. On the other hand if a specific slave needs exclusion
from synchronization the DBA would have to stop it manually which
would terminate its dump thread.

`mysqladmin shutdown' is extended with a `--wait_for_all_slaves' option
which translates to `SHUTDOW WAIT FOR ALL SLAVES' sql query
to enable the feature on the client side.

The patch also performs a small refactoring of the server shutdown
around close_connections() to introduce kill thread phases which
are two as of current.
2019-03-12 17:34:48 +02:00
Oleksandr Byelkin
6d68a3464e MDEV-18701: Wrong result from query that uses INTERSECT after UNION ALL
Recalculate distinct pointer if we cut chain of SELECTs
2019-03-11 17:41:00 +01:00
Alexander Barkov
5f34513c2a MDEV-18813 PROCEDURE and anonymous blocks silently ignore FETCH GROUP NEXT ROW
Part#2 (final): rewritting the code to pass the correct enum_sp_aggregate_type
to the sp_head constructor, so sp_head never changes its aggregation type
later on. The grammar has been simplified and defragmented.
This allowed to check aggregate specific instructions right after
a routine body has been scanned, by calling new LEX methods:
  sp_body_finalize_{procedure|function|trigger|event}()

Moving some C++ code from *.yy to a few new helper methods in LEX.
2019-03-07 18:48:15 +04:00
Alexander Barkov
a71d185a9a MDEV-18813 PROCEDURE and anonymous blocks silently ignore FETCH GROUP NEXT ROW
Part#1: moving opt_if_not_exists from "sf_tail" and "sp_tail" to "create".
2019-03-07 14:35:01 +04:00
Marko Mäkelä
2a791c53ad Merge 10.3 into 10.4 2019-03-06 09:00:52 +02:00
Marko Mäkelä
446b3ebdfc Merge 10.2 into 10.3
FIXME: Properly resolve conflicts between MDEV-18883
and MDEV-7742/MDEV-8305, and record the correct result for
main.log_slow
2019-03-05 12:56:05 +02:00
Marko Mäkelä
9835f7b80f Merge 10.1 into 10.2 2019-03-04 16:46:58 +02:00
Alexander Barkov
1bcb66c597 MDEV-18806 Synchronize ALTER TABLE EXCHANGE PARTITION and PURGE grammar in sql_yacc.yy and sql_yacc_ora.yy 2019-03-04 14:38:21 +04:00
Alexander Barkov
47b7ca629f MDEV-18796 Synchronize PS grammar between sql_yacc.yy and sql_yacc_ora.yy 2019-03-03 07:20:15 +04:00
Alexander Barkov
3b47587f41 MDEV-18789 Port "MDEV-7773 Aggregate stored functions" to sql_yacc_ora.yy 2019-03-01 23:55:55 +04:00
Oleksandr Byelkin
f2e1451740 Merge branch '10.0' into 10.1 2019-03-01 15:52:06 +01:00
Alexander Barkov
01d7727b76 MDEV-18779 Port sp_suid implementation from sql_yacc_ora.yy to sql_yacc.yy 2019-03-01 17:02:58 +04:00
Oleksandr Byelkin
7b5c63856b Merge branch '5.5' into 10.0 2019-02-28 21:50:00 +01:00
Oleksandr Byelkin
cb11b3fbe9 MDEV-17055: Server crashes in find_order_in_list upon 2nd (3rd) execution of SP with UPDATE
1. Always drop merged_for_insert flag on cleanup (there could be errors which prevent TABLE to be assigned)
2. Make more precise cleanup of select parts which was touched
2019-02-28 18:14:53 +01:00
Alexander Barkov
0ad598a00b A cleanup in derived table handling: removing duplicate code from st_select_lex::handle_derived()
st_select_lex::handle_derived() and mysql_handle_list_of_derived() had
exactly the same implementations.

- Adding a new method LEX::handle_list_of_derived() instead
- Removing public function mysql_handle_list_of_derived()
- Reusing LEX::handle_list_of_derived() in st_select_lex::handle_derived()
2019-02-28 18:13:28 +04:00
Alexander Barkov
09d29dfc76 MDEV-18767 Port "MDEV-16294: INSTALL PLUGIN IF NOT EXISTS / UNINSTALL PLUGIN IF EXISTS" to sql_yacc_ora.yy 2019-02-28 07:17:47 +04:00
Sergei Golubchik
1e6210161d MDEV-7597 Expiration of user passwords
post-merge changes:
* handle password expiration on old tables like everything else -
  make changes in memory, even if they cannot be done on disk
* merge "debug" tests with non-debug tests, they don't use dbug anyway
* only run rpl password expiration in MIXED mode, it doesn't replicate
  anything, so no need to repeat it thrice
* restore update_user_table_password() prototype, it should not change
  ACL_USER, this is done in acl_user_update()
* don't parse json twice in get_password_lifetime and get_password_expired
* remove LEX_USER::is_changing_password, see if there was any auth instead
* avoid overflow in expiration calculations
* don't initialize Account_options in the constructor, it's bzero-ed later
* don't create ulong sysvars - they're not portable, prefer uint or ulonglong
* misc simplifications
2019-02-21 15:04:03 +01:00
Robert Bindar
90ad4dbd17 MDEV-7597 Expiration of user passwords
This patch adds support for expiring user passwords.
The following statements are extended:
  CREATE USER user@localhost PASSWORD EXPIRE [option]
  ALTER USER user@localhost PASSWORD EXPIRE [option]
If no option is specified, the password is expired with immediate
effect. If option is DEFAULT, global policy applies according to
the default_password_lifetime system var (if 0, password never
expires, if N, password expires every N days). If option is NEVER,
the password never expires and if option is INTERVAL N DAY, the
password expires every N days.
The feature also supports the disconnect_on_expired_password system
var and the --connect-expired-password client option.

Closes #1166
2019-02-21 15:04:03 +01:00
Sergei Golubchik
81e4b9b3bb misc cleanups 2019-02-21 14:57:10 +01:00
Nikita Malyavin
6294516a56 MDEV-16975 Application-time periods: ALTER TABLE
* implicit period constraint is hidden and cannot be dropped independently
* create...like and create...select support
2019-02-21 14:57:09 +01:00
Nikita Malyavin
47e28a94d5 MDEV-16973 Application-time periods: DELETE
* inject portion of time updates into mysql_delete main loop
* triggered case emits delete+insert, no updates
* PORTION OF `SYSTEM_TIME` is forbidden
* `DELETE HISTORY .. FOR PORTION OF ...` is forbidden as well
2019-02-21 14:48:04 +01:00
Nikita Malyavin
073c93b194 MDEV-17082 Application-time periods: CREATE
* add syntax `CREATE TABLE ... PERIOD FOR <apptime>`
* add table period entity
2019-02-21 14:48:04 +01:00
Oleksandr Byelkin
93ac7ae70f Merge branch '10.3' into 10.4 2019-02-21 14:40:52 +01:00
Igor Babaev
4de3fd4ea2 MDEV-7486: Cosmetic changes
- Removed dead code
- Renamed a function
- Removed a parameter that not needed.
- Corrected comments
2019-02-18 22:11:25 -08:00
Galina Shalygina
7a77b221f1 MDEV-7486: Condition pushdown from HAVING into WHERE
Condition can be pushed from the HAVING clause into the WHERE clause
if it depends only on the fields that are used in the GROUP BY list
or depends on the fields that are equal to grouping fields.
Aggregate functions can't be pushed down.

How the pushdown is performed on the example:

SELECT t1.a,MAX(t1.b)
FROM t1
GROUP BY t1.a
HAVING (t1.a>2) AND (MAX(c)>12);

=>

SELECT t1.a,MAX(t1.b)
FROM t1
WHERE (t1.a>2)
GROUP BY t1.a
HAVING (MAX(c)>12);

The implementation scheme:

1. Extract the most restrictive condition cond from the HAVING clause of
   the select that depends only on the fields that are used in the GROUP BY
   list of the select (directly or indirectly through equalities)
2. Save cond as a condition that can be pushed into the WHERE clause
   of the select
3. Remove cond from the HAVING clause if it is possible

The optimization is implemented in the function
st_select_lex::pushdown_from_having_into_where().

New test file having_cond_pushdown.test is created.
2019-02-17 23:38:44 -08:00
Alexander Barkov
62c0ac2da6 A cleanup for MDEV-13916 Enforce check constraint on JSON type
1. Renaming Type_handler_json to Type_handler_json_longtext
   There will be other JSON handlers soon, e.g. Type_handler_json_varchar.

2. Making the code more symmetric for data types:

- Adding a new virtual method
  Type_handler::Column_definition_validate_check_constraint()
- Moving JSON-specific code from sql_yacc.yy to
  Type_handler_json_longtext::Column_definition_validate_check_constraint()

3. Adding new files sql_type_json.cc and sql_type_json.h
   and moving Type_handler+JSON related code into these files.
2019-02-15 19:46:58 +04:00
Sergei Golubchik
e5701d8363 cleanup: Account_options
move account options from LEX to Account_options structure
namely, mqh and ssl_*

Also, use LEX_CSTRING for ssl_*/x509_* strings and move
setting of ACL_USER::account_locked where it belongs
2019-02-14 17:30:22 +01:00
Robert Bindar
6c8ce999f8 MDEV-13095 Implement User Account locking
Add server support for user account locking.
This patch extends the ALTER/CREATE USER statements for
denying a user's subsequent login attempts:
  ALTER USER
    user [, user2] ACCOUNT [LOCK | UNLOCK]
  CREATE USER
    user [, user2] ACCOUNT [LOCK | UNLOCK]
The SHOW CREATE USER statement was updated to display the
locking state of an user.

Closes #1006
2019-02-14 17:30:22 +01:00
Igor Babaev
7d9f45e072 Merge branch '10.4' into bb-10.4-mdev17096 2019-02-13 14:59:34 -08:00
Monty
0f48949439 MDEV-13916 Enforce check constraint on JSON type
When creating a field of type JSON, it will be automatically
converted to TEXT with CHECK (json_valid(`a`)), if there wasn't any
previous check for the column.

Additional things:
- Added two bug fixes that was found while testing JSON. These bug
  fixes has also been pushed to 10.3 (with a test case), but as they
  where minimal and needed to get this task done and tested, the fixes
  are repeated here.
  - CREATE TABLE ... SELECT drops constraints for columns that
    are both in the create and select part.
  - If one has both a default expression and check constraint for a
    column, one can get the error "Expression for field `a` is refering
    to uninitialized field `a`.
- Removed some duplicate MYSQL_PLUGIN_IMPORT symbols
2019-02-13 19:40:26 +02:00
Igor Babaev
27c3abde30 MDEV-17096 Pushdown of simple derived tables to storage engines
MDEV-17631 select_handler for a full query pushdown

Added comments and file headers for files introduced in these tasks.
2019-02-12 22:56:24 -08:00
Oleksandr Byelkin
65c5ef9b49 dirty merge 2019-02-07 13:59:31 +01:00
Igor Babaev
3f9040085a Merge branch '10.4' into bb-10.4-mdev17096 2019-02-06 18:01:29 -08:00
Igor Babaev
16327fc2e7 MDEV-17096 Pushdown of simple derived tables to storage engines
MDEV-17631 select_handler for a full query pushdown

Interfaces + Proof of Concept for federatedx with test cases.

The interfaces have been developed for integration of ColumnStore engine.
2019-02-06 17:02:44 -08:00
Marko Mäkelä
081fd8bfa2 Merge 10.1 into 10.2 2019-02-02 11:40:02 +02:00
Andrei Elkin
5d48ea7d07 MDEV-10963 Fragmented BINLOG query
The problem was originally stated in
  http://bugs.mysql.com/bug.php?id=82212
The size of an base64-encoded Rows_log_event exceeds its
vanilla byte representation in 4/3 times.
When a binlogged event size is about 1GB mysqlbinlog generates
a BINLOG query that can't be send out due to its size.

It is fixed with fragmenting the BINLOG argument C-string into
(approximate) halves when the base64 encoded event is over 1GB size.
The mysqlbinlog in such case puts out

    SET @binlog_fragment_0='base64-encoded-fragment_0';
    SET @binlog_fragment_1='base64-encoded-fragment_1';
    BINLOG @binlog_fragment_0, @binlog_fragment_1;

to represent a big BINLOG.
For prompt memory release BINLOG handler is made to reset the BINLOG argument
user variables in the middle of processing, as if @binlog_fragment_{0,1} = NULL
is assigned.

Notice the 2 fragments are enough, though the client and server still may
need to tweak their @@max_allowed_packet to satisfy to the fragment
size (which they would have to do anyway with greater number of
fragments, should that be desired).

On the lower level the following changes are made:

Log_event::print_base64()
  remains to call encoder and store the encoded data into a cache but
  now *without* doing any formatting. The latter is left for time
  when the cache is copied to an output file (e.g mysqlbinlog output).
  No formatting behavior is also reflected by the change in the meaning
  of the last argument which specifies whether to cache the encoded data.

Rows_log_event::print_helper()
  is made to invoke a specialized fragmented cache-to-file copying function
  which is

copy_cache_to_file_wrapped()
  that takes care of fragmenting also optionally wraps encoded
  strings (fragments) into SQL stanzas.

my_b_copy_to_file()
  is refactored to into my_b_copy_all_to_file(). The former function
  is generalized
  to accepts more a limit argument to constraint the copying and does
  not reinitialize anymore the cache into reading mode.
  The limit does not do any effect on the fully read cache.
2019-01-24 20:44:50 +02:00
Vladislav Vaintroub
4bcd7da076 fixup! Fixed compiler warnings from optimized builds 2019-01-15 16:44:13 +01:00
Michael Widenius
9990027f87 Fixed compiler warnings from optimized builds 2019-01-15 17:05:42 +02:00
Monty
c53aab974b Added syntax and implementation for BACKUP STAGE's
Part of MDEV-5336 Implement LOCK FOR BACKUP

- Changed check of Global_only_lock to also include BACKUP lock.
- We store latest MDL_BACKUP_DDL lock in thd->mdl_backup_ticket to be able
  to downgrade lock during copy_data_between_tables()
2018-12-09 22:12:27 +02:00
Marko Mäkelä
dde2ca4aa1 Merge 10.3 into 10.4 2018-11-19 20:22:33 +02:00
Alexander Barkov
62bcd74712 MDEV-17694 Add method LEX::sp_proc_stmt_statement_finalize() 2018-11-14 07:38:28 +04:00
Alexander Barkov
2a0b6de41b MDEV-17253 Oracle compatibility: The REVERSE key word for FOR loop behaves incorrectly 2018-11-13 18:03:14 +04:00
Alexander Barkov
f5855ba03d MDEV-17664 Add sql_mode specific tokens for ':' and '%' 2018-11-11 09:35:05 +04:00
Marko Mäkelä
c761b43451 Merge 10.3 into 10.4 2018-11-08 10:19:55 +02:00
Oleksandr Byelkin
54b2e1c1be MDEV-16697: Fix difference between 32bit/windows and 64bit systems in allowed select nest level 2018-11-07 09:43:12 +01:00
Marko Mäkelä
074c684099 Merge 10.3 into 10.4 2018-11-06 16:24:16 +02:00