<?xml version="1.0" encoding="utf-8"?>
<rss xmlns:atom="http://www.w3.org/2005/Atom" version="2.0">
    <channel>
        <title>Abul成长之路</title>
        <link>http://www.tusundong.top</link>
        <description>Abul的个人博客</description>
        <atom:link href="http://www.tusundong.top/rss.html" rel="self" />
        <language>zh-cn</language>
        <lastBuildDate>Fri, 11 Sep 2026 10:15:42 GMT</lastBuildDate>
        <item>
            <title>mysql docker镜像冷知识</title>
            <link>http://www.tusundong.top/post/mysql docker-jing-xiang-leng-zhi-shi.html</link>
            <description><![CDATA[
            <div class="toc"><ul>
<li><a href="#toc-409">缺失工具列表（默认 oracle‑linux 镜像）</a></li>
</ul>
</div><p>Docker 官方 mysql 镜像从 <strong>8.0.29</strong> 开始，<strong>默认 tag（不带 - debian 后缀）底层切换为 Oracle Linux 8 的最小化 RPM 包</strong>，只安装 <code>mysql-community-server</code>，不再安装 <code>mysql-community-client</code> 完整客户端包GitHub。</p>
<ul>
<li><code>mysqlbinlog</code>、<code>mysqlcheck</code> 这些工具<strong>属于 mysql‑community-client 包，不属于 server 包</strong>。</li>
<li>Oracle Linux RPM：<code>mysql‑community‑server</code> 只提供<code>mysqld</code>服务程序；客户端工具独立在<code>mysql‑community-client</code>包。</li>
<li>镜像构建脚本为缩小镜像体积，默认不再安装完整 client 包，因此丢失 mysqlbinlog 等工具GitHub。</li>
<li><strong><code>mysql:8.0.x‑debian</code> 后缀镜像不受影响</strong>：Debian 系 deb 包中 server 包会自带部分客户端工具，<code>mysqlbinlog</code>依然存在GitHub</li>
</ul>
<h2><a id="toc-409" class="anchor" href="#toc-409"></a>缺失工具列表（默认 oracle‑linux 镜像）</h2>
<pre><code class="hljs lang-ebnf"><span class="hljs-attribute">mysqlbinlog
mysqlcheck
myisamchk
myisam_ftdump
myisampack
myisamlog</span>
</code></pre>

            ]]></description>
            <pubDate>Thu, 03 Sep 2026 07:31:26 GMT</pubDate>
            <guid>http://www.tusundong.top/post/mysql docker-jing-xiang-leng-zhi-shi.html</guid>
        </item>
        <item>
            <title>MySQL 26.7.0版本</title>
            <link>http://www.tusundong.top/post/MySQL 26.7.0-ban-ben.html</link>
            <description><![CDATA[
            <div class="toc"><ul>
<li><a href="#toc-e5e">一、MySQL 26.7.0 到底是什么</a></li>
<li><a href="#toc-b79">二、为什么版本号直接跳到 26，而不是 9.8？</a></li>
<li><a href="#toc-104">三、相对 MySQL 9.7 新增的核心功能（26.7.0 Innovation）</a><ul>
<li><a href="#toc-8ea">1、复制：Change Stream Applier（CSA）‑全新回放引擎</a><ul>
<li><a href="#toc-3d0">2、线程池 Thread‑Pool 插件下放社区版（重磅福利）</a></li>
<li><a href="#toc-5ec">3、安全：TLS 后量子密码学 (PQC) 密钥交换支持</a></li>
<li><a href="#toc-c44">4、审计日志 Audit‑Log 新增监控指标</a></li>
<li><a href="#toc-cae">5、InnoDB 底层基础设施增强（用户无感知，稳定性优化）</a></li>
</ul>
</li>
</ul>
</li>
</ul>
</div><h2><a id="toc-e5e" class="anchor" href="#toc-e5e"></a>一、MySQL 26.7.0 到底是什么</h2>
<p>一段时间没关注mysql版本发布，最近竟然出了个<a href="https://dev.mysql.com/doc/refman/26.7/en/">MySQL 26.7.0版本</a>。</p>
<p><img src="https://www.tusundong.top/static/upload/20260826/upload_3a469a435272c1b37ce7430e839b7066.png" alt="image.png"></p>
<p>好奇，为什么一下子从9.7升级到26.7。AI了下，MySQL 26.7.0 = 2026‑07 发布的第一个 <a href="https://blogs.oracle.com/mysql/mysql-july-2026-ga-releases-now-available?f_link_type=f_linkinlinenote&flow_extra=eyJkb2NfcG9zaXRpb24iOjAsImRvY19pZCI6IjMwODY5ZWRkOTc4MDgyOWEtMmRjOTRhN2IwYWMxNThlMiIsImlubGluZV9kaXNwbGF5X3Bvc2l0aW9uIjowfQ%3D%3D">Innovation（创新版）</a>，是 MySQL 切换到【日历版本 CalVer】后的首个正式 GA 版本。  </p>
<blockquote>
<p>版本格式：YY.M.P = 年份后两位。发布月份。补丁号<br>26 → 2026 年<br>7 → 7 月<br>0 → 第 0 号初始补丁版本</p>
</blockquote>
<h2><a id="toc-b79" class="anchor" href="#toc-b79"></a>二、为什么版本号直接跳到 26，而不是 9.8？</h2>
<p>旧模式：语义版本 Major.Minor.Patch（5.7→8.0→8.4→9.7），版本号看不出发布时间，生命周期难判断。<br>新模式：日历版本 CalVer，直接用「年份‑月份」当版本前缀，一眼就能知道什么时候发布。</p>
<h2><a id="toc-104" class="anchor" href="#toc-104"></a>三、相对 MySQL 9.7 新增的核心功能（26.7.0 Innovation）</h2>
<h3><a id="toc-8ea" class="anchor" href="#toc-8ea"></a>1、复制：Change Stream Applier（CSA）‑全新回放引擎</h3>
<blockquote>
<p>可选择开启，老的 MTA 多线程回放器仍然保留     </p>
</blockquote>
<ul>
<li>每个复制通道独立配置 worker 并发数，不再全局统一</li>
<li>单通道 worker 范围：1‑1024</li>
<li>事务执行与提交排序逻辑解耦，并行回放能力更强</li>
<li>新增单通道事件内存上限控制参数</li>
<li>STOP REPLICA 响应速度更快</li>
<li>Performance‑Schema 可观测每个通道 worker 运行状态</li>
<li>开启方式：CHANGE REPLICATION SOURCE TO ... APPLIER_VERSION=2;<br> APPLIER_VERSION=1 = 传统 MTA 模式（默认）</li>
</ul>
<p>新增系统变量：  </p>
<ul>
<li>replica_csa_workers</li>
<li>replica_csa_event_memory_limit</li>
</ul>
<h4><a id="toc-3d0" class="anchor" href="#toc-3d0"></a>2、线程池 Thread‑Pool 插件下放社区版（重磅福利）</h4>
<ul>
<li>历史：线程池插件一直仅企业版可用</li>
<li>26.7 变更：社区版内置 thread_pool 插件，开箱可用</li>
<li>作用：不再是一线程一连接模型；高并发短连接场景降低上下文切换、减少 InnoDB 资源争抢，稳定吞吐。</li>
</ul>
<h4><a id="toc-5ec" class="anchor" href="#toc-5ec"></a>3、安全：TLS 后量子密码学 (PQC) 密钥交换支持</h4>
<p>前提：编译链接 OpenSSL 3.5+</p>
<ul>
<li>支持混合后量子密钥交换套件（X25519MLKEM768、secp384r1MLKEM768、secp384r1MLKEM1024）</li>
<li>覆盖：客户端连接、管理端口、X‑Protocol、主从复制链路</li>
<li>新增十余项 PQC 相关系统变量，支持强制 / 优先后量子算法、降级回退传统 RSA‑ECC</li>
<li>目的：抵御未来量子计算机破解现有 TLS 密钥</li>
</ul>
<h4><a id="toc-c44" class="anchor" href="#toc-c44"></a>4、审计日志 Audit‑Log 新增监控指标</h4>
<p>新增状态变量，用于 JSON 格式审计日志：</p>
<ul>
<li>audit_log_file_count（plugin）</li>
<li>audit_log.file_count（component）<br>实时统计当前审计日志文件数量，监控轮转与清理任务。</li>
</ul>
<h4><a id="toc-cae" class="anchor" href="#toc-cae"></a>5、InnoDB 底层基础设施增强（用户无感知，稳定性优化）</h4>
<ul>
<li>undo log 截断（undo truncation）流程重构，移除本地标记文件，改用 undo 页头标记状态，崩溃恢复更加健壮</li>
<li>Redo‑log、表空间、MVCC/ReadView、IX 锁恢复等底层接口解耦重构<blockquote>
<p>不改变 SQL 语义、MVCC 可见性行为，无新对外 SQL 功能，为后续 InnoDB 新特性铺路。</p>
</blockquote>
</li>
</ul>

            ]]></description>
            <pubDate>Wed, 26 Aug 2026 08:40:54 GMT</pubDate>
            <guid>http://www.tusundong.top/post/MySQL 26.7.0-ban-ben.html</guid>
        </item>
        <item>
            <title>MySQL一些特殊锁等待</title>
            <link>http://www.tusundong.top/post/MySQL-yi-xie-te-shu-suo-deng-dai.html</link>
            <description><![CDATA[
            <div class="toc"><ul>
<li><a href="#toc-85f">1. Waiting for global read lock</a></li>
<li><a href="#toc-5e6">2. Waiting for backup lock</a></li>
<li><a href="#toc-914">3. Waiting for table metadata lock</a></li>
</ul>
</div><blockquote>
<p>数据库卡住了，最容易排查的就是慢SQL阻塞，processlist排序即可；次之，则是长事务未提交，导致锁等待，这种查询information_schema.innodb_trx按照trx_started排序即可找出根源；但往往有一些锁等待，前面两种方式很难定位出来。</p>
</blockquote>
<h2><a id="toc-85f" class="anchor" href="#toc-85f"></a>1. Waiting for global read lock</h2>
<p>触发：有线程执行flush table with read lock(FTWRL)，mysqldump、xtrabackup都可能触发</p>
<pre><code class="hljs lang-n1ql"># 获取全局锁来源
<span class="hljs-keyword">select</span> OBJECT_TYPE,OBJECT_SCHEMA,OBJECT_NAME,LOCK_STATUS,
 (<span class="hljs-keyword">SELECT</span> PROCESSLIST_ID <span class="hljs-keyword">FROM</span> performance_schema.threads <span class="hljs-keyword">WHERE</span> THREAD_ID = t.OWNER_THREAD_ID) <span class="hljs-keyword">AS</span> pid,
 (<span class="hljs-keyword">SELECT</span> PROCESSLIST_USER <span class="hljs-keyword">FROM</span> performance_schema.threads <span class="hljs-keyword">WHERE</span> THREAD_ID = t.OWNER_THREAD_ID) <span class="hljs-keyword">AS</span> <span class="hljs-keyword">user</span> 
<span class="hljs-keyword">FROM</span> performance_schema.metadata_locks <span class="hljs-keyword">AS</span> t <span class="hljs-keyword">where</span> OBJECT_TYPE=<span class="hljs-string">&#x27;GLOBAL&#x27;</span> <span class="hljs-keyword">AND</span> LOCK_STATUS=<span class="hljs-string">&#x27;GRANTED&#x27;</span>;

# KILL上面的pid
</code></pre>
<h2><a id="toc-5e6" class="anchor" href="#toc-5e6"></a>2. Waiting for backup lock</h2>
<p>触发：物理冷备（xtrabackup、mysqlbackup）获取一致性快照，LOCK INSTANCE FOR BACKUP</p>
<pre><code class="hljs lang-pgsql"># 查询BACKUP <span class="hljs-keyword">LOCK</span>
<span class="hljs-keyword">select</span> OBJECT_TYPE,OBJECT_SCHEMA,OBJECT_NAME,LOCK_STATUS,
 (<span class="hljs-keyword">SELECT</span> PROCESSLIST_ID <span class="hljs-keyword">FROM</span> performance_schema.threads <span class="hljs-keyword">WHERE</span> THREAD_ID = t.OWNER_THREAD_ID) <span class="hljs-keyword">AS</span> pid,
 (<span class="hljs-keyword">SELECT</span> PROCESSLIST_USER <span class="hljs-keyword">FROM</span> performance_schema.threads <span class="hljs-keyword">WHERE</span> THREAD_ID = t.OWNER_THREAD_ID) <span class="hljs-keyword">AS</span> <span class="hljs-keyword">user</span> 
<span class="hljs-keyword">FROM</span> performance_schema.metadata_locks <span class="hljs-keyword">AS</span> t <span class="hljs-keyword">where</span> OBJECT_TYPE=<span class="hljs-string">&#x27;BACKUP LOCK&#x27;</span> <span class="hljs-keyword">AND</span> LOCK_STATUS=<span class="hljs-string">&#x27;GRANTED&#x27;</span>;

# KILL上面的pid
</code></pre>
<h2><a id="toc-914" class="anchor" href="#toc-914"></a>3. Waiting for table metadata lock</h2>
<p>触发：普通查询、变更都会持有表的MDL，长事务、lock tables t read</p>
<pre><code class="hljs lang-pgsql"># 找出被阻塞的block_processlist_id
<span class="hljs-keyword">SELECT</span>
  <span class="hljs-comment">-- 锁对象公共信息</span>
  w.lock_object_type   <span class="hljs-keyword">AS</span> object_type,
  w.table_schema       <span class="hljs-keyword">AS</span> table_schema,
  w.<span class="hljs-built_in">table_name</span>         <span class="hljs-keyword">AS</span> <span class="hljs-built_in">table_name</span>,
  w.metadata_lock_type <span class="hljs-keyword">AS</span> lock_type,
  <span class="hljs-comment">-- 等待锁的线程（被阻塞）</span>
  w.lock_status        <span class="hljs-keyword">AS</span> status,
  w.connection_id      <span class="hljs-keyword">AS</span> processlist_id,
  w.client_ip          <span class="hljs-keyword">AS</span> client_ip,
  concat(left(w.current_sql,<span class="hljs-number">20</span>),<span class="hljs-string">&#x27;...&#x27;</span>)        <span class="hljs-keyword">AS</span> `<span class="hljs-keyword">sql</span>`,
  <span class="hljs-comment">-- 持有锁的线程（阻塞源）</span>
  g.lock_status        <span class="hljs-keyword">AS</span> hold_status,
  g.connection_id      <span class="hljs-keyword">AS</span> &quot;block_processlist_id(*)&quot;,
  g.client_ip          <span class="hljs-keyword">AS</span> block_client_ip
  ,concat(left(g.current_sql,<span class="hljs-number">20</span>),<span class="hljs-string">&#x27;...&#x27;</span>)       <span class="hljs-keyword">AS</span> block_sql
<span class="hljs-keyword">FROM</span>
(
  <span class="hljs-keyword">SELECT</span> 
    ml.OBJECT_TYPE       <span class="hljs-keyword">AS</span> lock_object_type,
    ml.OBJECT_SCHEMA     <span class="hljs-keyword">AS</span> table_schema,
    ml.OBJECT_NAME       <span class="hljs-keyword">AS</span> <span class="hljs-built_in">table_name</span>,
    ml.LOCK_TYPE         <span class="hljs-keyword">AS</span> metadata_lock_type,
    ml.LOCK_STATUS       <span class="hljs-keyword">AS</span> lock_status,
    ml.LOCK_DURATION     <span class="hljs-keyword">AS</span> lock_duration,
    t.PROCESSLIST_ID     <span class="hljs-keyword">AS</span> connection_id,
    t.PROCESSLIST_HOST   <span class="hljs-keyword">AS</span> client_ip,
    COALESCE(esc.SQL_TEXT, t.PROCESSLIST_INFO) <span class="hljs-keyword">AS</span> current_sql
  <span class="hljs-keyword">FROM</span> performance_schema.metadata_locks ml
  <span class="hljs-keyword">JOIN</span> performance_schema.threads t <span class="hljs-keyword">ON</span> ml.OWNER_THREAD_ID = t.THREAD_ID
  <span class="hljs-keyword">LEFT JOIN</span> performance_schema.events_statements_current esc <span class="hljs-keyword">ON</span> t.THREAD_ID = esc.THREAD_ID
  <span class="hljs-keyword">WHERE</span> ml.LOCK_STATUS = <span class="hljs-string">&#x27;PENDING&#x27;</span>
    <span class="hljs-keyword">AND</span> ml.OBJECT_TYPE <span class="hljs-keyword">IN</span> (<span class="hljs-string">&#x27;TABLE&#x27;</span>,<span class="hljs-string">&#x27;SCHEMA&#x27;</span>) 
    <span class="hljs-keyword">AND</span> t.PROCESSLIST_ID <span class="hljs-keyword">IS</span> <span class="hljs-keyword">NOT</span> <span class="hljs-keyword">NULL</span>
) w
<span class="hljs-keyword">JOIN</span>
(
  <span class="hljs-keyword">SELECT</span> 
    ml.OBJECT_TYPE       <span class="hljs-keyword">AS</span> lock_object_type,
    ml.OBJECT_SCHEMA     <span class="hljs-keyword">AS</span> table_schema,
    ml.OBJECT_NAME       <span class="hljs-keyword">AS</span> <span class="hljs-built_in">table_name</span>,
    ml.LOCK_TYPE         <span class="hljs-keyword">AS</span> metadata_lock_type,
    ml.LOCK_STATUS       <span class="hljs-keyword">AS</span> lock_status,
    ml.LOCK_DURATION     <span class="hljs-keyword">AS</span> lock_duration,
    t.PROCESSLIST_ID     <span class="hljs-keyword">AS</span> connection_id,
    t.PROCESSLIST_HOST   <span class="hljs-keyword">AS</span> client_ip,
    COALESCE(esc.SQL_TEXT, t.PROCESSLIST_INFO) <span class="hljs-keyword">AS</span> current_sql
  <span class="hljs-keyword">FROM</span> performance_schema.metadata_locks ml
  <span class="hljs-keyword">JOIN</span> performance_schema.threads t <span class="hljs-keyword">ON</span> ml.OWNER_THREAD_ID = t.THREAD_ID
  <span class="hljs-keyword">LEFT JOIN</span> performance_schema.events_statements_current esc <span class="hljs-keyword">ON</span> t.THREAD_ID = esc.THREAD_ID
  <span class="hljs-keyword">WHERE</span> ml.LOCK_STATUS = <span class="hljs-string">&#x27;GRANTED&#x27;</span>
    <span class="hljs-keyword">AND</span> ml.OBJECT_TYPE <span class="hljs-keyword">IN</span> (<span class="hljs-string">&#x27;TABLE&#x27;</span>,<span class="hljs-string">&#x27;SCHEMA&#x27;</span>) 
    <span class="hljs-keyword">AND</span> t.PROCESSLIST_ID <span class="hljs-keyword">IS</span> <span class="hljs-keyword">NOT</span> <span class="hljs-keyword">NULL</span>
) g
<span class="hljs-keyword">ON</span> w.lock_object_type = g.lock_object_type 
<span class="hljs-keyword">AND</span> w.table_schema = g.table_schema 
<span class="hljs-keyword">AND</span> w.<span class="hljs-built_in">table_name</span> = g.<span class="hljs-built_in">table_name</span> 
<span class="hljs-keyword">AND</span> w.lock_duration = g.lock_duration
<span class="hljs-keyword">AND</span> w.connection_id != g.connection_id
<span class="hljs-keyword">ORDER</span> <span class="hljs-keyword">BY</span> w.connection_id;

# kill上一步的block_processlist_id
</code></pre>

            ]]></description>
            <pubDate>Tue, 16 Jun 2026 01:52:19 GMT</pubDate>
            <guid>http://www.tusundong.top/post/MySQL-yi-xie-te-shu-suo-deng-dai.html</guid>
        </item>
        <item>
            <title>神奇的MDL</title>
            <link>http://www.tusundong.top/post/shen-qi-de-MDL.html</link>
            <description><![CDATA[
            <div class="toc"><ul>
<li><a href="#toc-114">问题复现</a></li>
</ul>
</div><p>早上RD突然发消息说，有个DDL<code>alter table xx modify column col varchar(10)</code>无法执行。<br>一开始以为是个简单的长事务没有关闭，对表xx加了S锁。熟练的查询<code>select * from information_schema.innodb_trx;</code>竟然没有数据！再查一下processlist，alter table xx的进程真的是出于Waiting for table metadata lock。此时判断可能真没长事务，但有高频查询SQL ，导致DDL无法获取到X锁。看了下监控指标，查询QPS竟然也不高...此时跟RD反馈，让他把本地的SQL查询工具关了。alter执行成功！反馈说有开了另外一个窗口进行事务insert，但执行报错。  </p>
<h3><a id="toc-114" class="anchor" href="#toc-114"></a>问题复现</h3>
<pre><code class="hljs lang-sql">会话一：
<span class="hljs-keyword">create</span> <span class="hljs-keyword">table</span> tmp(id <span class="hljs-type">int</span>,val <span class="hljs-type">varchar</span>(<span class="hljs-number">5</span>));
<span class="hljs-keyword">begin</span>;
<span class="hljs-keyword">insert</span> <span class="hljs-keyword">into</span> tmp <span class="hljs-keyword">values</span>(<span class="hljs-number">1</span>,<span class="hljs-string">&#x27;aaaaaa&#x27;</span>);  <span class="hljs-comment">/*执行报错，事务没关闭*/</span>

会话二：
<span class="hljs-keyword">alter</span> <span class="hljs-keyword">table</span> tmp modify <span class="hljs-keyword">column</span> val <span class="hljs-type">varchar</span>(<span class="hljs-number">10</span>); <span class="hljs-comment">/*MDL锁等待*/</span>

会话三：
<span class="hljs-keyword">select</span> <span class="hljs-operator">*</span> <span class="hljs-keyword">from</span> information_schema.innodb_trx; <span class="hljs-comment">/*查无会话一对应的长事务*/</span>
<span class="hljs-keyword">select</span> <span class="hljs-operator">*</span> <span class="hljs-keyword">from</span> performance_schema.metadata_locks;
</code></pre>
<p>疑问：</p>
<ol>
<li>会话三为何查无会话一的长事务，因为innodb_trx仅记录已成功开始执行的活跃事务，会话一开始就执行失败。</li>
<li>如何定位到是会话一导致的？（开启PS，select * from performance_schema.metadata_locks）</li>
</ol>

            ]]></description>
            <pubDate>Tue, 30 Sep 2025 03:29:22 GMT</pubDate>
            <guid>http://www.tusundong.top/post/shen-qi-de-MDL.html</guid>
        </item>
        <item>
            <title>grafana sqlite3转mysql</title>
            <link>http://www.tusundong.top/post/grafana sqlite3-zhuan-mysql.html</link>
            <description><![CDATA[
            <div class="toc"><ul>
<li><a href="#toc-8e1">背景</a></li>
<li><a href="#toc-4b8">解决</a><ul>
<li><a href="#toc-216">迁移过程的参考步骤​</a></li>
</ul>
</li>
</ul>
</div><blockquote>
<p>引用自<a href="https://blog.yousri.org/posts/change-grafana-db-to-mysql.html">迁移Grafana 默认的 SQLite DB 数据改为使用 MySQL 方式</a></p>
</blockquote>
<h3><a id="toc-8e1" class="anchor" href="#toc-8e1"></a>背景</h3>
<p>grafana默认使用sqlite3存储数据，数据量上升需调整为MySQL，涉及将SQLite数据转MySQL。
一开始搜索文档为sqlite to mysql，各种折腾，比较有参考意义的，如：<a href="https://study-database.readthedocs.io/zh/latest/%E6%95%B0%E6%8D%AE%E5%BA%9304sqlite%E8%BD%ACmysql.html#sqlitesql">https://study-database.readthedocs.io/zh/latest/%E6%95%B0%E6%8D%AE%E5%BA%9304sqlite%E8%BD%ACmysql.html#sqlitesql</a> ，总结了各种方案，遇到的坑，虽然最后也没采用。如果直接转换，涉及字段类型text可能需转换成varchar，或者text，因此表结构最好直接通过grafana来生成。</p>
<h3><a id="toc-4b8" class="anchor" href="#toc-4b8"></a>解决</h3>
<h4><a id="toc-216" class="anchor" href="#toc-216"></a>迁移过程的参考步骤​</h4>
<ul>
<li>首先需要暂停 Grafana 服务，导出默认的 SQLite 数据库文件，如 grafana.db ；</li>
<li>可借助 Grafana 官方提供的工具 database-migrator 将数据库导出转化为支持 MySQL 协议的 SQL 文件；</li>
<li>修改 Grafana 配置文件 grafana.ini 数据库相关配置参数 -e &quot;GF_DATABASE_URL=mysql://$db_username:$db_passwd@$db_host:$db_port/$db_name&quot; 使用 MySQL 数据库；</li>
<li>重新启动 Grafana 让其在 MySQL 中设置数据库和表结构，初始化完再先关停下 Grafana ；</li>
<li>导入前面使用工具从 SQLLite 原生默认数据库中导出的 sql 数据到 MySQL 对应的 DB 中；</li>
<li>重新开启 Grafana 按理即可；</li>
</ul>
<p>附：</p>
<ol>
<li>grafana官方文档：<a href="https://grafana.com/docs/grafana/latest/setup-grafana/installation/docker/">https://grafana.com/docs/grafana/latest/setup-grafana/installation/docker/</a></li>
<li>database-migrator：<a href="https://github.com/grafana/database-migrator">https://github.com/grafana/database-migrator</a></li>
</ol>

            ]]></description>
            <pubDate>Tue, 11 Mar 2025 07:06:46 GMT</pubDate>
            <guid>http://www.tusundong.top/post/grafana sqlite3-zhuan-mysql.html</guid>
        </item>
        <item>
            <title>MySQL8异常中断Forcing close of thread 24 user: xxx</title>
            <link>http://www.tusundong.top/post/MySQL8-yi-chang-zhong-duan-Forcing close of thread 24 user: xxx.html</link>
            <description><![CDATA[
            <div class="toc"><ul>
<li><a href="#toc-6ba">ubuntu自动升级一些事</a></li>
</ul>
</div><ol>
<li>环境：ubuntu22.04 mysql8.0.39</li>
<li>问题：mysql服务突然停止shutdown，查看mysql异常日志如下：</li>
</ol>
<pre><code class="hljs lang-crmsh">Received SHUTDOWN from <span class="hljs-keyword">user</span> <span class="hljs-title">&lt;via</span> <span class="hljs-keyword">user</span> <span class="hljs-title">signal</span>&gt;. Shutting down mysqld (<span class="hljs-keyword">Version</span>: <span class="hljs-number">8.0</span>.<span class="hljs-number">39</span>-<span class="hljs-number">0</span>ubuntu0.<span class="hljs-number">22.04</span>.<span class="hljs-number">1</span>).
/usr/sbin/mysqld: Forcing close of thread <span class="hljs-number">24</span>  user: &#x27;zabbix&#x27;.
/usr/sbin/mysqld: Shutdown complete (mysqld <span class="hljs-number">8.0</span>.<span class="hljs-number">39</span>-<span class="hljs-number">0</span>ubuntu0.<span class="hljs-number">22.04</span>.<span class="hljs-number">1</span>)  (Ubuntu).
</code></pre>
<ol start="3">
<li>排查：</li>
</ol>
<ul>
<li>查资料，有的说是DNS反向解析导致mysql异常停止，要开启skip-name-resolve，查my.cnf已开启。</li>
<li>查看/var/log/syslog和journalctl -xe<br>  发现每次mysql停止前都会出现启动<strong>apt-daily-upgrade.service</strong>，而该服务是每天定时更新服务upgrade and clean activities，再查/var/log/apt/term.log也看到了确实是更新的时候触发mysql shutdown，也有出现mysql被直接升级到最新</li>
</ul>
<ol start="4">
<li>解决：<br>  卸载mysql服务（当时dpkg安装），重新安装；或者停止apt-daily-upgrade</li>
</ol>
<h3><a id="toc-6ba" class="anchor" href="#toc-6ba"></a>ubuntu自动升级一些事</h3>
<ol>
<li>触发服务apt-daily和apt-daily-upgrade<br>systemctl status apt-daily-upgrade查看，都是调用同一个脚本/usr/lib/apt/apt.systemd.daily</li>
<li>何为apt-daily-upgrade<br>顾名思义就是自动升级。<br>  是否开启自动升级、自动升级规则，查看目录/etc/apt/apt.conf.d/<br>  重点关注50unattended-upgrades（无人值守自动升级），默认都是开启，每日6点多默默后台升级</li>
<li>查看日志<br>已自动升级日志：/var/log/apt/history.log<br>升级日志明细（含kill）：/var/log/apt/term.log</li>
<li>自动升级规则<br>/etc/apt/apt.conf.d/50unattended-upgrades</li>
</ol>
<pre><code class="hljs lang-nsis"> Unattended-<span class="hljs-class">Upgrade::Allowed</span>-Origins {
      <span class="hljs-string">&quot;<span class="hljs-variable">${distro_id}</span>:<span class="hljs-variable">${distro_codename}</span>&quot;</span><span class="hljs-comment">;</span>
      <span class="hljs-string">&quot;<span class="hljs-variable">${distro_id}</span>:<span class="hljs-variable">${distro_codename}</span>-security&quot;</span><span class="hljs-comment">;</span>
      <span class="hljs-string">&quot;<span class="hljs-variable">${distro_id}</span>ESMApps:<span class="hljs-variable">${distro_codename}</span>-apps-security&quot;</span><span class="hljs-comment">;</span>
      <span class="hljs-string">&quot;<span class="hljs-variable">${distro_id}</span>ESM:<span class="hljs-variable">${distro_codename}</span>-infra-security&quot;</span><span class="hljs-comment">;</span>
 }<span class="hljs-comment">;</span>
</code></pre>
<p>  ${distro_id}代表发行版的ID（如Ubuntu）；<br>  ${distro_codename}代表发行版的代号（如jammy，对应于Ubuntu 22.04）</p>
<pre><code class="hljs lang-pgsql">例如：`apt <span class="hljs-keyword">search</span> mysql-<span class="hljs-keyword">server</span>` （如下版本就支持自动升级）  
mysql-<span class="hljs-keyword">server</span><span class="hljs-number">-8.0</span>/jammy-updates,jammy-<span class="hljs-keyword">security</span> <span class="hljs-number">8.0</span><span class="hljs-number">.37</span><span class="hljs-number">-0</span>ubuntu0<span class="hljs-number">.22</span><span class="hljs-number">.04</span><span class="hljs-number">.3</span> amd64
</code></pre>
<ol start="5">
<li>已安装包查看是否会被自动升级<br><code>apt policy mysql-server</code></li>
</ol>
<pre><code class="hljs lang-apache"><span class="hljs-attribute">mysql</span>-server:
  <span class="hljs-attribute">Installed</span>: (none)						(已安装版本)
  <span class="hljs-attribute">Candidate</span>: <span class="hljs-number">8</span>.<span class="hljs-number">0</span>.<span class="hljs-number">37</span>-<span class="hljs-number">0</span>ubuntu<span class="hljs-number">0.22.04.3</span>	(候选版本)
  <span class="hljs-attribute">Version</span> table:						(版本表)
     <span class="hljs-attribute">8</span>.<span class="hljs-number">0</span>.<span class="hljs-number">37</span>-<span class="hljs-number">0</span>ubuntu<span class="hljs-number">0.22.04.3</span> <span class="hljs-number">500</span>
        <span class="hljs-attribute">500</span> http://cn.archive.ubuntu.com/ubuntu jammy-updates/main amd<span class="hljs-number">64</span> Packages
        <span class="hljs-attribute">500</span> http://cn.archive.ubuntu.com/ubuntu jammy-security/main amd<span class="hljs-number">64</span> Packages
     <span class="hljs-attribute">8</span>.<span class="hljs-number">0</span>.<span class="hljs-number">28</span>-<span class="hljs-number">0</span>ubuntu<span class="hljs-number">4</span> <span class="hljs-number">500</span>
        <span class="hljs-attribute">500</span> http://cn.archive.ubuntu.com/ubuntu jammy/main amd<span class="hljs-number">64</span> Packages
</code></pre>
<ol start="6">
<li>经验<br>因此，如果使用apt安装来自ubuntu官方源的包，可能会被静默升级，导致服务不可用。</li>
</ol>

            ]]></description>
            <pubDate>Sat, 10 Aug 2024 06:28:27 GMT</pubDate>
            <guid>http://www.tusundong.top/post/MySQL8-yi-chang-zhong-duan-Forcing close of thread 24 user: xxx.html</guid>
        </item>
        <item>
            <title>零宽字符导致MySQL表神奇消失</title>
            <link>http://www.tusundong.top/post/ling-kuan-zi-fu-dao-zhi-MySQL-biao-shen-qi-xiao-shi.html</link>
            <description><![CDATA[
            <div class="toc"><ul>
<li><a href="#toc-90c">复现</a></li>
<li><a href="#toc-21d">思考</a></li>
</ul>
</div><p>很久没写文章了，今天看了篇文章《<a href="https://opensource.actionsky.com/%e6%8a%80%e6%9c%af%e5%88%86%e4%ba%ab-mysql-%e5%88%9b%e5%bb%ba%e8%a1%a8%e5%90%8e%e7%a5%9e%e7%a7%98%e6%b6%88%e5%a4%b1%ef%bc%9f%e6%8f%ad%e7%a7%98%e9%9b%b6%e5%ae%bd%e5%ad%97%e7%ac%a6%e9%99%b7%e9%98%b1/">MySQL 创建表后神秘消失？揭秘零宽字符陷阱</a>》，一开始也是被标题吸引进去。  </p>
<h3><a id="toc-90c" class="anchor" href="#toc-90c"></a>复现</h3>
<pre><code class="hljs lang-sql"><span class="hljs-keyword">CREATE</span> <span class="hljs-keyword">TABLE</span> `lang​` (
  `id` <span class="hljs-type">int</span>(<span class="hljs-number">11</span>) <span class="hljs-keyword">NOT</span> <span class="hljs-keyword">NULL</span> AUTO_INCREMENT COMMENT <span class="hljs-string">&#x27;主键自增&#x27;</span>,
  `create_time` datetime <span class="hljs-keyword">NOT</span> <span class="hljs-keyword">NULL</span> <span class="hljs-keyword">DEFAULT</span> <span class="hljs-built_in">CURRENT_TIMESTAMP</span> COMMENT <span class="hljs-string">&#x27;创建时间&#x27;</span>,
  <span class="hljs-keyword">PRIMARY</span> KEY (`id`)
) ENGINE<span class="hljs-operator">=</span>InnoDB <span class="hljs-keyword">DEFAULT</span> CHARSET<span class="hljs-operator">=</span>utf8;

# <span class="hljs-keyword">show</span> tables;可看到表lang，但直接<span class="hljs-keyword">select</span>查不到
<span class="hljs-keyword">select</span> <span class="hljs-operator">*</span> <span class="hljs-keyword">from</span> lang;
ERROR <span class="hljs-number">1146</span> (<span class="hljs-number">42</span>S02): <span class="hljs-keyword">Table</span> <span class="hljs-string">&#x27;test.lang&#x27;</span> doesn<span class="hljs-string">&#x27;t exist
</span></code></pre>
<p>原因为lang后面含有零宽字符
<img src="https://www.tusundong.top/static/upload/20240430/upload_b4bf15f4a4c4b2c1a66abe739c6cfd76.png" alt="image.png"></p>
<h3><a id="toc-21d" class="anchor" href="#toc-21d"></a>思考</h3>
<ol>
<li>原文中已经说了如何解决，例如复制到编辑器中可能会提示异常符号、show tables查看的时候边框线 “|” 与其他行是不对齐的。  </li>
<li>而我的排查方式不一样，第一直觉就是觉得里面含有特殊字符，直接查hex字符。</li>
</ol>
<pre><code class="hljs lang-sql">abul test<span class="hljs-operator">&gt;</span><span class="hljs-keyword">select</span> hex(table_name),hex(<span class="hljs-string">&#x27;lang&#x27;</span>) <span class="hljs-keyword">from</span> information_schema.tables <span class="hljs-keyword">where</span> table_schema<span class="hljs-operator">=</span><span class="hljs-string">&#x27;test&#x27;</span> <span class="hljs-keyword">and</span> table_namelike <span class="hljs-string">&#x27;lang%&#x27;</span>;
<span class="hljs-operator">+</span><span class="hljs-comment">-----------------+-------------+</span>
<span class="hljs-operator">|</span> hex(table_name) <span class="hljs-operator">|</span> hex(<span class="hljs-string">&#x27;lang&#x27;</span>) <span class="hljs-operator">|</span>
<span class="hljs-operator">+</span><span class="hljs-comment">-----------------+-------------+</span>
<span class="hljs-operator">|</span> <span class="hljs-number">6</span>C616E67E2808B  <span class="hljs-operator">|</span> <span class="hljs-number">6</span>C616E67    <span class="hljs-operator">|</span>
<span class="hljs-operator">+</span><span class="hljs-comment">-----------------+-------------+</span>
<span class="hljs-number">1</span> <span class="hljs-type">row</span> <span class="hljs-keyword">in</span> <span class="hljs-keyword">set</span> (<span class="hljs-number">0.00</span> sec)
</code></pre>
<ol start="3">
<li>另外，如何避免？上线SQL正常会经过goInception审核，那如何避免RD提交类似特殊字符。可能得正则查找。</li>
</ol>
<pre><code class="hljs lang-undefined">get_first_zero_width_char_pos(sql:str) -&gt; int:
    &quot;&quot;&quot;
    查找第一个零宽字符的位置，无零宽字符返回-1
    &quot;&quot;&quot;
    zwc = re.search(r&#x27;[\u200B-\u200D\u2060\uFEFF]&#x27;, sql)
    return zwc.start() if zwc else -1
</code></pre>
<ol start="4">
<li>零宽字符大多数情况不可见，那会用在什么地方？某些社交平台，敏感词汇会被脱敏，这样是否就可以反脱敏。</li>
</ol>

            ]]></description>
            <pubDate>Tue, 30 Apr 2024 01:36:13 GMT</pubDate>
            <guid>http://www.tusundong.top/post/ling-kuan-zi-fu-dao-zhi-MySQL-biao-shen-qi-xiao-shi.html</guid>
        </item>
        <item>
            <title>DTS同步引发的存储过程执行报错</title>
            <link>http://www.tusundong.top/post/DTS-tong-bu-yin-fa-de-cun-chu-guo-cheng-zhi-xing-bao-cuo.html</link>
            <description><![CDATA[
            <div class="toc"><ul>
<li><a href="#toc-90c">复现</a></li>
<li><a href="#toc-772">排查</a><ul>
<li><a href="#toc-5e1">1. 权限</a></li>
<li><a href="#toc-393">2. 手动执行</a></li>
<li><a href="#toc-f94">3. 比对存储过程</a></li>
</ul>
</li>
<li><a href="#toc-54b">结论</a></li>
</ul>
</div><p>场景：今天OP反馈使用DTS将zabbix同步到新实例，原先旧实例执行存储过程<code>call partition_maintenance_all(&#39;zabbix&#39;)</code>成功，迁移到新实例后，执行报错，提示如下：</p>
<pre><code class="hljs lang-pgsql"><span class="hljs-keyword">ALTER</span> command denied <span class="hljs-keyword">to</span> <span class="hljs-keyword">user</span> <span class="hljs-string">&#x27;db_arch_w&#x27;</span>@<span class="hljs-string">&#x27;x.x.x.x&#x27;</span> <span class="hljs-keyword">for</span> <span class="hljs-keyword">table</span> <span class="hljs-string">&#x27;history&#x27;</span>
</code></pre>
<h3><a id="toc-90c" class="anchor" href="#toc-90c"></a>复现</h3>
<pre><code class="hljs lang-vala"><span class="hljs-meta"># 旧实例，执行成功</span>
mysql -h [旧实例地址] -u db_arch_w -p zabbix -e <span class="hljs-string">&quot;call partition_maintenance_all(&#x27;zabbix&#x27;);&quot;</span>

<span class="hljs-meta"># 新实例，执行报错，提示`ALTER command denied`</span>
mysql -h [新实例地址] -u db_arch_w -p zabbix -e <span class="hljs-string">&quot;call partition_maintenance_all(&#x27;zabbix&#x27;);&quot;</span>
</code></pre>
<h3><a id="toc-772" class="anchor" href="#toc-772"></a>排查</h3>
<h4><a id="toc-5e1" class="anchor" href="#toc-5e1"></a>1. 权限</h4>
<p>提示没权限，首先比对下新旧实例db_arch_w的权限，竟然都一样，如下，且没有zabbix库的ALTER权限！ </p>
<pre><code class="hljs lang-n1ql">show grants for &#x27;db_arch_w&#x27;@&#x27;%&#x27;;
+---------------------------------------------------------------------------------------------------------------------------------+
| Grants for db_arch_w@%                                                                                                          |
+---------------------------------------------------------------------------------------------------------------------------------+
| GRANT <span class="hljs-keyword">SELECT</span>, <span class="hljs-keyword">INSERT</span>, <span class="hljs-keyword">UPDATE</span>, <span class="hljs-keyword">DELETE</span>, <span class="hljs-keyword">CREATE</span>, REPLICATION SLAVE, REPLICATION CLIENT <span class="hljs-keyword">ON</span> *.* <span class="hljs-keyword">TO</span> <span class="hljs-symbol">`db_arch_w`</span>@<span class="hljs-symbol">`%`</span> <span class="hljs-keyword">WITH</span> <span class="hljs-keyword">GRANT</span> <span class="hljs-keyword">OPTION</span> |
| <span class="hljs-keyword">GRANT</span> SHOW_ROUTINE <span class="hljs-keyword">ON</span> *.* <span class="hljs-keyword">TO</span> <span class="hljs-symbol">`db_arch_w`</span>@<span class="hljs-symbol">`%`</span> <span class="hljs-keyword">WITH</span> <span class="hljs-keyword">GRANT</span> <span class="hljs-keyword">OPTION</span>                                                                  |
| <span class="hljs-keyword">GRANT</span> <span class="hljs-keyword">EXECUTE</span> <span class="hljs-keyword">ON</span> <span class="hljs-symbol">`zabbix`</span>.* <span class="hljs-keyword">TO</span> <span class="hljs-symbol">`db_arch_w`</span>@<span class="hljs-symbol">`%`</span>                                                                                  |
+---------------------------------------------------------------------------------------------------------------------------------+
</code></pre>
<p>确实没有<code>ALTER</code>权限，但旧实例为什么能执行？</p>
<h4><a id="toc-393" class="anchor" href="#toc-393"></a>2. 手动执行</h4>
<p>看了下报错的位置，手动执行，执行SQL如下</p>
<pre><code class="hljs lang-vala"><span class="hljs-meta"># 新实例，alter报错，提示ALTER command denied</span>
mysql -h [新实例地址] -u db_arch_w -p zabbix -e <span class="hljs-string">&quot;alter table zabbix.history add partition (partition p202403260000 values less than(1711468800))&quot;</span>

<span class="hljs-meta"># 旧实例，一样报错</span>
mysql -h [旧实例地址] -u db_arch_w -p zabbix -e <span class="hljs-string">&quot;alter table zabbix.history add partition (partition p202403260000 values less than(1711468800))&quot;</span>
</code></pre>
<p>这个时候新旧报错又是一致的，这是为什么呢？</p>
<h4><a id="toc-f94" class="anchor" href="#toc-f94"></a>3. 比对存储过程</h4>
<p>其实在第2步的时候，就可以确定是存储过程有问题。</p>
<pre><code class="hljs lang-n1ql"># 旧实例存储过程
<span class="hljs-keyword">CREATE</span> DEFINER=<span class="hljs-symbol">`admin`</span>@<span class="hljs-symbol">`%`</span> <span class="hljs-keyword">PROCEDURE</span> <span class="hljs-symbol">`partition_maintenance_all`</span>(
<span class="hljs-keyword">IN</span> <span class="hljs-symbol">`SCHEMA_NAME`</span> varchar(<span class="hljs-number">32</span>) charset utf8mb4 <span class="hljs-keyword">collate</span> utf8mb4_0900_ai_ci )
<span class="hljs-keyword">BEGIN</span>
    # xxx 具体内容，忽略
<span class="hljs-keyword">END</span>


# 新实例存储过程
<span class="hljs-keyword">CREATE</span> DEFINER=<span class="hljs-symbol">`admin`</span>@<span class="hljs-symbol">`%`</span> <span class="hljs-keyword">PROCEDURE</span> <span class="hljs-symbol">`partition_maintenance_all`</span>(
<span class="hljs-keyword">IN</span> <span class="hljs-symbol">`SCHEMA_NAME`</span> varchar(<span class="hljs-number">32</span>) charset utf8mb4 <span class="hljs-keyword">collate</span> utf8mb4_0900_ai_ci )
    SQL SECURITY INVOKER
<span class="hljs-keyword">BEGIN</span>
    # xxx 具体内容，忽略
<span class="hljs-keyword">END</span>
</code></pre>
<p>有个细微区别，新实例存储过程中含有<code>SQL SECURITY INVOKER</code>，其含义为执行权限使用<strong>调用者</strong>账号所属权限。例如我们一开始用<code>db_arch_w</code>账号连接，则使用<code>db_arch_w</code>的权限去执行，而存储过程中含有ALTER命令，db_arch_w没有权限就报错；而旧实例没有指定SQL SECURITY，则默认使用DEFINER定义者的权限，此处为admin账号权限（超管），因此执行不会报错。</p>
<blockquote>
<p>SQL SECURITY：有两个值DEFINER和INVOKER，没有指定则默认DEFINER。<br> DEFINER：使用定义者权限；INVOKER：使用调用者权限。</p>
</blockquote>
<h3><a id="toc-54b" class="anchor" href="#toc-54b"></a>结论</h3>
<p>DTS同步时，存储过程会自动添加<code>SQL SECURITY INVOKER</code>，导致调用时使用的是调用者的权限，当存储过程中含有调用者没有的权限时就会报错。因此仅需删除所有存储过程中<code>SQL SECURITY INVOKER</code>即可执行。</p>
<p>思考：为什么DTS要指定INVOKER呢？可能是考虑同步的目标实例没有定义的用户，会导致无法执行。</p>

            ]]></description>
            <pubDate>Mon, 11 Mar 2024 04:39:43 GMT</pubDate>
            <guid>http://www.tusundong.top/post/DTS-tong-bu-yin-fa-de-cun-chu-guo-cheng-zhi-xing-bao-cuo.html</guid>
        </item>
        <item>
            <title>浅谈MySQL json类型</title>
            <link>http://www.tusundong.top/post/mysql_json.html</link>
            <description><![CDATA[
            <div class="toc"><ul>
<li><a href="#toc-75e">浅谈MySQL json类型</a><ul>
<li><a href="#toc-63c">一、官网说明</a></li>
<li><a href="#toc-63a">二、使用说明</a></li>
<li><a href="#toc-5d2">三、建议</a></li>
</ul>
</li>
</ul>
</div><h1><a id="toc-75e" class="anchor" href="#toc-75e"></a>浅谈MySQL json类型</h1>
<blockquote>
<p>mysql很早就已经支持原生的json数据类型（从MySQL 5.7.8起），但目前看还是有些开发同学<strong>不了解</strong>，或者<strong>不敢用</strong>json类型。下面我们一起来学习下官网mysql的json类型。
{.is-info}</p>
</blockquote>
<h2><a id="toc-63c" class="anchor" href="#toc-63c"></a>一、官网说明</h2>
<blockquote>
<p><code>As of MySQL 5.7.8, MySQL supports a native JSON data type defined by RFC 7159 that enables efficient access to data in JSON (JavaScript Object Notation) documents. The JSON data type provides these advantages over storing JSON-format strings in a string column:</code>
从MySQL5.7.8开始，mysql支持RFC 7159定义的原生JSON数据类型，能够有效访问JSON文档中的数据。相比于存储于字符串，json类型存储有以下优点：<br/>
  <code>Automatic validation of JSON documents stored in JSON columns. Invalid documents produce an error.</code>
  自动校验json文档是否正确，不合法的值将返回error。<br/>
  <code>Optimized storage format. JSON documents stored in JSON columns are converted to an internal format that permits quick read access to document elements. When the server later must read a JSON value stored in this binary format, the value need not be parsed from a text representation. The binary format is structured to enable the server to look up subobjects or nested values directly by key or array index without reading all values before or after them in the document.</code>
  优化存储格式。存储在json列中的json文档会被转换成内部格式，使得更快访问文档里面的元素。当服务器需要读取以二进制格式存储的JSON值时，不需要从文本中解析。二进制格式结构化能时服务器直接通过键值或者数组索引查找对应值，而不需要访问之前或之后的所有值。<br/>
<code>The space required to store a JSON document is roughly the same as for LONGBLOB or LONGTEXT; see Section 11.7, “Data Type Storage Requirements”, for more information. It is important to keep in mind that the size of any JSON document stored in a JSON column is limited to the value of the max_allowed_packet system variable.</code>
JSON文档占用的空间与LONGBLOB或者LONGTEXT基本等同。JSON列的大小受限于max_allowed_packet变量的值。<br/>
<code>A JSON column cannot have a non-NULL default value.</code>
JSON类型不能有非空默认值。<br/>
<code>JSON columns, like columns of other binary types, are not indexed directly; instead, you can create an index on a generated column that extracts a scalar value from the JSON column. </code>
JSON字段跟其它二进制类型一样，不能直接创建索引；但是可以通过在生成的列上创建一个索引，生成的列指向JSON里的标量值。</p>
</blockquote>
<p>参考文档：<a href="https://dev.mysql.com/doc/refman/5.7/en/json.html">https://dev.mysql.com/doc/refman/5.7/en/json.html</a></p>
<p><strong>总结：</strong></p>
<ol>
<li><p>mysql5.7.8开始支持json类型（开发、测试、生产MySQL版本5.7.18，支持√）；  </p>
</li>
<li><p>相比于使用varchar存储，json类型能自动校验json格式是否正确；</p>
</li>
<li><p>能直接访问json对象里的键值对；</p>
</li>
<li><p>占用空间大，能存储大字段
  理论能存储<code>字符数num = max_allowed_packet / 单字符大小(字母1，中文3，emoji4)</code>  </p>
</li>
<li><p>JSON类型不能有非空默认值；</p>
</li>
</ol>
<pre><code class="hljs lang-sql"><span class="hljs-keyword">create</span> <span class="hljs-keyword">table</span> t_json(
  json_wrong json <span class="hljs-keyword">not</span> <span class="hljs-keyword">null</span> <span class="hljs-keyword">default</span> <span class="hljs-string">&#x27;{}&#x27;</span>,   <span class="hljs-comment">--&gt; 这是错的，不能有非空默认值</span>
  json_right json <span class="hljs-keyword">null</span> <span class="hljs-keyword">default</span> <span class="hljs-keyword">null</span>        <span class="hljs-comment">--&gt; 这是允许的，default null</span>
)
</code></pre>
<ol start="6">
<li>json字段不能直接建索引，但可以通过创建虚拟列，指向JSON里的对应key来创建索引。</li>
</ol>
<h2><a id="toc-63a" class="anchor" href="#toc-63a"></a>二、使用说明</h2>
<pre><code class="hljs lang-sql"><span class="hljs-comment">-- 创建表</span>
<span class="hljs-keyword">create</span> <span class="hljs-keyword">table</span> t_json(
  id <span class="hljs-type">int</span> unsigned <span class="hljs-keyword">not</span> <span class="hljs-keyword">null</span> auto_increment,
  json_data json <span class="hljs-keyword">null</span> <span class="hljs-keyword">default</span> <span class="hljs-keyword">null</span> comment <span class="hljs-string">&#x27;json类型字段&#x27;</span>,
  <span class="hljs-keyword">primary</span> key(id)
)comment <span class="hljs-string">&#x27;创建json类型表&#x27;</span>;

<span class="hljs-comment">-- 插入数据，跟字符串一样</span>
<span class="hljs-keyword">insert</span> <span class="hljs-keyword">into</span> t_json(json_data) <span class="hljs-keyword">values</span>(<span class="hljs-string">&#x27;{&quot;id&quot;:1, &quot;name&quot;:&quot;abul&quot;}&#x27;</span>);

<span class="hljs-comment">-- 查询，使用-&gt;或者-&gt;&gt;，注意两个区别（附1）</span>
<span class="hljs-keyword">select</span> json_data<span class="hljs-operator">-</span><span class="hljs-operator">&gt;</span>&quot;$.name&quot; <span class="hljs-keyword">as</span> name <span class="hljs-keyword">from</span> t_json;
<span class="hljs-operator">+</span><span class="hljs-comment">--------+</span>
<span class="hljs-operator">|</span> name   <span class="hljs-operator">|</span>
<span class="hljs-operator">+</span><span class="hljs-comment">--------+</span>
<span class="hljs-operator">|</span> &quot;abul&quot; <span class="hljs-operator">|</span>
<span class="hljs-operator">+</span><span class="hljs-comment">--------+</span>

<span class="hljs-keyword">select</span> json_data<span class="hljs-operator">-</span><span class="hljs-operator">&gt;&gt;</span>&quot;$.name&quot; <span class="hljs-keyword">as</span> name <span class="hljs-keyword">from</span> t_json;
<span class="hljs-operator">+</span><span class="hljs-comment">------+</span>
<span class="hljs-operator">|</span> name <span class="hljs-operator">|</span>
<span class="hljs-operator">+</span><span class="hljs-comment">------+</span>
<span class="hljs-operator">|</span> abul <span class="hljs-operator">|</span>
<span class="hljs-operator">+</span><span class="hljs-comment">------+</span>

<span class="hljs-comment">-- 根据json里的字段模糊搜索，其余操作相同</span>
<span class="hljs-keyword">select</span> <span class="hljs-operator">*</span> <span class="hljs-keyword">from</span> t_json <span class="hljs-keyword">where</span> json_data<span class="hljs-operator">-</span><span class="hljs-operator">&gt;&gt;</span>&quot;$.name&quot; <span class="hljs-keyword">like</span> <span class="hljs-string">&#x27;ab%&#x27;</span>;

<span class="hljs-comment">-- 变更某个key值</span>
update t_json <span class="hljs-keyword">set</span> json_data<span class="hljs-operator">=</span>json_replace(json_data,<span class="hljs-string">&#x27;$.name&#x27;</span>,<span class="hljs-string">&#x27;xmabul&#x27;</span>) <span class="hljs-keyword">where</span> id<span class="hljs-operator">=</span><span class="hljs-number">1</span>;

<span class="hljs-comment">-- 创建虚拟列和索引</span>
<span class="hljs-comment">-- 场景：json某个key为高频查询，需要创建key对应索引</span>
<span class="hljs-keyword">create</span> <span class="hljs-keyword">table</span> t_json_index(
  id <span class="hljs-type">int</span> unsigned <span class="hljs-keyword">not</span> <span class="hljs-keyword">null</span> auto_increment,
  json_data json <span class="hljs-keyword">null</span> <span class="hljs-keyword">default</span> <span class="hljs-keyword">null</span> comment <span class="hljs-string">&#x27;json类型字段&#x27;</span>,
  name <span class="hljs-type">varchar</span>(<span class="hljs-number">20</span>) generated always <span class="hljs-keyword">as</span> (json_data<span class="hljs-operator">-</span><span class="hljs-operator">&gt;&gt;</span>&quot;$.name&quot;) comment <span class="hljs-string">&#x27;json.name&#x27;</span>,  <span class="hljs-comment">-- 虚拟列</span>
  <span class="hljs-keyword">primary</span> key(id),
  index idx_name(name)   <span class="hljs-comment">-- 虚拟列索引</span>
)comment <span class="hljs-string">&#x27;创建json类型表索引&#x27;</span>;

<span class="hljs-comment">-- 判断指定值是否在json数组中</span>
<span class="hljs-keyword">where</span> json_contains(clear_rule_json,<span class="hljs-built_in">json_object</span>(&quot;typeEnum&quot;,<span class="hljs-string">&#x27;MERCHANT_GET&#x27;</span>))


<span class="hljs-comment">-- json函数</span>
<span class="hljs-comment">-- 详细文档：https://dev.mysql.com/doc/refman/5.7/en/json-function-reference.html</span>
<span class="hljs-operator">-</span><span class="hljs-operator">&gt;</span>										等价于JSON_EXTRACT().		
<span class="hljs-operator">-</span><span class="hljs-operator">&gt;&gt;</span>										等价于JSON_UNQUOTE(JSON_EXTRACT())	
<span class="hljs-built_in">JSON_ARRAY</span>()					创建一个数组
JSON_ARRAY_APPEND()		扩展json数组对象		
JSON_ARRAY_INSERT()		在指定数组位置插入值 
JSON_CONTAINS()				判断指定值是否在json文档中		
JSON_CONTAINS_PATH()	判断是否包含路径	
JSON_DEPTH()					返回最深层级		
JSON_EXTRACT()				解析json，返回<span class="hljs-keyword">value</span>值		
JSON_INSERT()					插入值		
JSON_KEYS()						返回所有key		
JSON_LENGTH()					返回json元素个数		
JSON_MERGE()					json合并
JSON_MERGE_PATCH()		合并json，替换重复key值	
JSON_MERGE_PRESERVE()	json合并，相同key合并值	
<span class="hljs-built_in">JSON_OBJECT</span>()					创建一个object对象		
JSON_PRETTY()					美化json对象
JSON_QUOTE()					添加引号		
JSON_REMOVE()					移除json中的某个key		
JSON_REPLACE()				替换json某个key值		
JSON_SEARCH()					查找值对应路径		
JSON_SET()						插入key值，相当于replace <span class="hljs-keyword">into</span>		
JSON_STORAGE_SIZE()		返回存储json文档的二进制的字节数
JSON_TYPE()						返回类型		
JSON_UNQUOTE()				取消引号		
JSON_VALID()					判断json是否合法		
</code></pre>
<p>附：</p>
<ol>
<li>-&gt;和-&gt;&gt;区别
mysql 5.7.9开始支持-&gt; ;mysql5.7.13开始支持-&gt;&gt;<br>-&gt;等价于JSON_EXTRACT()，将json拆解，如果是字符串会包含引号
-&gt;&gt;等价于JSON_UNQUOTE()，将json拆解，并且去掉引号</li>
</ol>
<h2><a id="toc-5d2" class="anchor" href="#toc-5d2"></a>三、建议</h2>
<ol>
<li>如果存储的json数据长度小于1000，并且不需要通过json文档里的key进行查询过滤，建议直接定义varchar类型，减少占用空间。</li>
<li>如果需要根据键值查询，可以定义成json；但是当需要存储大字段时，还是建议存储到mongodb中，专业的存储引擎做专业的事。</li>
</ol>
<p>原则上json数据存储建议 <strong>mongodb &gt; mysql.varchar &gt; mysql.json</strong></p>

            ]]></description>
            <pubDate>Fri, 06 Jan 2023 12:54:17 GMT</pubDate>
            <guid>http://www.tusundong.top/post/mysql_json.html</guid>
        </item>
        <item>
            <title>MySQL内存计算</title>
            <link>http://www.tusundong.top/post/MySQL-nei-cun-ji-suan.html</link>
            <description><![CDATA[
            <div class="toc"><ul>
<li><a href="#toc-dd4">1. 最大可使用内存（ M ）：</a></li>
<li><a href="#toc-2bc">2.单个连接最大可使用内存（ M ）：</a></li>
<li><a href="#toc-09b">3.最大可使用内存（不包含连接占用内存）（ M ）：</a></li>
</ul>
</div><h3><a id="toc-dd4" class="anchor" href="#toc-dd4"></a>1. 最大可使用内存（ M ）：</h3>
<pre><code class="hljs lang-sql"><span class="hljs-keyword">SELECT</span> 
  (
    @<span class="hljs-variable">@key</span>_buffer_size <span class="hljs-operator">+</span> 
    @<span class="hljs-variable">@query</span>_cache_size <span class="hljs-operator">+</span> 
    @<span class="hljs-variable">@tmp</span>_table_size <span class="hljs-operator">+</span> 
    @<span class="hljs-variable">@innodb</span>_buffer_pool_size  <span class="hljs-operator">+</span> 
    @<span class="hljs-variable">@innodb</span>_log_buffer_size <span class="hljs-operator">+</span> 
    @<span class="hljs-variable">@max</span>_connections <span class="hljs-operator">*</span> (
        @<span class="hljs-variable">@sort</span>_buffer_size <span class="hljs-operator">+</span> 
        @<span class="hljs-variable">@read</span>_buffer_size <span class="hljs-operator">+</span> 
        @<span class="hljs-variable">@read</span>_rnd_buffer_size <span class="hljs-operator">+</span> 
        @<span class="hljs-variable">@join</span>_buffer_size <span class="hljs-operator">+</span> 
        @<span class="hljs-variable">@binlog</span>_cache_size <span class="hljs-operator">+</span> 
        @<span class="hljs-variable">@thread</span>_stack
    )
  ) <span class="hljs-operator">/</span> <span class="hljs-number">1024</span> <span class="hljs-operator">/</span> <span class="hljs-number">1024</span>  <span class="hljs-keyword">as</span> `MaxMem(M)` ;
</code></pre>
<h3><a id="toc-2bc" class="anchor" href="#toc-2bc"></a>2.单个连接最大可使用内存（ M ）：</h3>
<pre><code class="hljs lang-sql">  <span class="hljs-keyword">SELECT</span> 
 (
     @<span class="hljs-variable">@sort</span>_buffer_size <span class="hljs-operator">+</span> 
     @<span class="hljs-variable">@read</span>_buffer_size <span class="hljs-operator">+</span> 
     @<span class="hljs-variable">@read</span>_rnd_buffer_size <span class="hljs-operator">+</span> 
     @<span class="hljs-variable">@join</span>_buffer_size <span class="hljs-operator">+</span> 
     @<span class="hljs-variable">@binlog</span>_cache_size <span class="hljs-operator">+</span> 
     @<span class="hljs-variable">@thread</span>_stack
 ) <span class="hljs-operator">/</span> <span class="hljs-number">1024</span> <span class="hljs-operator">/</span> <span class="hljs-number">1024</span> <span class="hljs-keyword">AS</span> `ConnectMem(M)` ;
</code></pre>
<h3><a id="toc-09b" class="anchor" href="#toc-09b"></a>3.最大可使用内存（不包含连接占用内存）（ M ）：</h3>
<pre><code class="hljs lang-less">   <span class="hljs-selector-tag">SELECT</span> 
     (
   <span class="hljs-variable">@@key_buffer_size</span> + 
   <span class="hljs-variable">@@query_cache_size</span> + 
   <span class="hljs-variable">@@tmp_table_size</span> + 
   <span class="hljs-variable">@@innodb_buffer_pool_size</span>  + 
   <span class="hljs-variable">@@innodb_log_buffer_size</span>
     ) / <span class="hljs-selector-tag">1024</span> / <span class="hljs-selector-tag">1024</span> <span class="hljs-selector-tag">AS</span> `<span class="hljs-selector-tag">MaxMemExcCon</span>(M)`;
</code></pre>

            ]]></description>
            <pubDate>Wed, 21 Dec 2022 02:45:10 GMT</pubDate>
            <guid>http://www.tusundong.top/post/MySQL-nei-cun-ji-suan.html</guid>
        </item>
    </channel>
</rss>
