微信公众号搜"智元新知"关注
微信扫一扫可直接关注哦！

从两个不同的表中按列排序时,PostgreSQL查询的速度会变慢

分类：Postgre SQL作者：编程之家
以前,我使用了这个查询,速度很快： 

cb=# explain analyze SELECT "web_route"."id","web_crag"."id" FROM "web_route" 
INNER JOIN "web_crag" ON ( "web_route"."crag_id" = "web_crag"."id" )
WHERE "web_crag"."type" IN (1,2) 
ORDER BY "web_crag"."name" ASC
LIMIT 20;
                                                                 QUERY PLAN                                                                  
---------------------------------------------------------------------------------------------------------------------------------------------
 Limit  (cost=0.00..2.16 rows=20 width=18) (actual time=0.027..0.105 rows=20 loops=1)
   ->  nested Loop  (cost=0.00..47088.94 rows=436055 width=18) (actual time=0.026..0.100 rows=20 loops=1)
         ->  Index Scan using web_crag_name on web_crag  (cost=0.00..503.16 rows=1776 width=14) (actual time=0.011..0.020 rows=14 loops=1)
               Filter: (type = ANY ('{1,2}'::integer[]))
         ->  Index Scan using web_route_crag_id on web_route  (cost=0.00..23.27 rows=296 width=8) (actual time=0.004..0.005 rows=1 loops=14)
               Index Cond: (crag_id = web_crag.id)
 Total runtime: 0.154 ms
(7 rows)

查询的问题在于返回行的顺序不确定,这导致后续页面中的重复行产生OFFSETing(即分页在我的Web应用程序中无法正常工作).我决定通过“web_route”.id“进行额外排序来使排序严格.

cb=# explain analyze SELECT "web_route"."id",2)
ORDER BY "web_crag"."name","web_route"."id" ASC 
LIMIT 20;
                                                             QUERY PLAN                                                             
------------------------------------------------------------------------------------------------------------------------------------
 Limit  (cost=29189.04..29189.09 rows=20 width=18) (actual time=324.065..324.068 rows=20 loops=1)
   ->  Sort  (cost=29189.04..30279.18 rows=436055 width=18) (actual time=324.063..324.064 rows=20 loops=1)
         Sort Key: web_crag.name,web_route.id
         Sort Method: top-N heapsort  Memory: 26kB
         ->  Hash Join  (cost=135.40..17585.78 rows=436055 width=18) (actual time=0.882..195.941 rows=435952 loops=1)
               Hash Cond: (web_route.crag_id = web_crag.id)
               ->  Seq Scan on web_route  (cost=0.00..10909.55 rows=436055 width=8) (actual time=0.026..55.916 rows=435952 loops=1)
               ->  Hash  (cost=113.20..113.20 rows=1776 width=14) (actual time=0.848..0.848 rows=1775 loops=1)
                     Buckets: 1024  Batches: 1  Memory Usage: 82kB
                     ->  Seq Scan on web_crag  (cost=0.00..113.20 rows=1776 width=14) (actual time=0.004..0.510 rows=1775 loops=1)
                           Filter: (type = ANY ('{1,2}'::integer[]))
 Total runtime: 324.101 ms
(12 rows)

但是,正如您所看到的,查询速度超过2000x,这是相当多的:).我想知道如果有的话可以做些什么.我打算做一个非常好的黑客并将“web_crag”.“name”复制到“web_route”中,以便我可以在两列(crag_name,id)上放一个索引,但如果有更好的方法我会很高兴.

以下是“web_route”和“web_crag”的方案,以防万一.

cb=# \d web_crag;
                                      Table "public.web_crag"
     Column      |           Type           |                       Modifiers                       
-----------------+--------------------------+-------------------------------------------------------
 id              | integer                  | not null default nextval('web_crag_id_seq'::regclass)
 name            | character varying(64)    | not null
 latitude        | double precision         | 
 longitude       | double precision         | 
 type            | integer                  | 
 description     | text                     | not null
 normalized_name | character varying(64)    | not null
 country_id      | integer                  | 
 location_index  | character(24)            | not null
 added_by_id     | integer                  | 
 date_created    | timestamp with time zone | 
 last_modified   | timestamp with time zone | 
Indexes:
    "web_crag_pkey" PRIMARY KEY,btree (id)
    "web_crag_added_by_id" btree (added_by_id)
    "web_crag_country_id" btree (country_id)
    "web_crag_location_index" btree (location_index)
    "web_crag_name" btree (name)
Foreign-key constraints:
    "added_by_id_refs_id_1745ebe43b31bec6" FOREIGN KEY (added_by_id) REFERENCES web_member(id) DEFERRABLE INITIALLY DEFERRED
    "country_id_refs_id_1384050a9bd763af" FOREIGN KEY (country_id) REFERENCES web_country(id) DEFERRABLE INITIALLY DEFERRED
Referenced by:
    TABLE "web_route" CONSTRAINT "crag_id_refs_id_3ce1145606d12740" FOREIGN KEY (crag_id) REFERENCES web_crag(id) DEFERRABLE INITIALLY DEFERRED
    TABLE "web_video" CONSTRAINT "crag_id_refs_id_4fc9cbf2832725ca" FOREIGN KEY (crag_id) REFERENCES web_crag(id) DEFERRABLE INITIALLY DEFERRED
    TABLE "web_image" CONSTRAINT "crag_id_refs_id_58210dd331468848" FOREIGN KEY (crag_id) REFERENCES web_crag(id) DEFERRABLE INITIALLY DEFERRED
    TABLE "web_eventdestination" CONSTRAINT "crag_id_refs_id_612ad57c4d76c32c" FOREIGN KEY (crag_id) REFERENCES web_crag(id) DEFERRABLE INITIALLY DEFERRED
Triggers:
    set_crag_location_index BEFORE INSERT OR UPDATE ON web_crag FOR EACH ROW EXECUTE PROCEDURE set_crag_location_index()

cb=# \d web_route
                                        Table "public.web_route"
       Column       |           Type           |                       Modifiers                        
--------------------+--------------------------+--------------------------------------------------------
 id                 | integer                  | not null default nextval('web_route_id_seq'::regclass)
 name               | character varying(64)    | not null
 crag_id            | integer                  | not null
 sector             | character varying(64)    | not null
 difficulty         | character varying(16)    | not null
 author             | character varying(64)    | not null
 build_date         | character varying(32)    | not null
 description        | text                     | not null
 difficulty_numeric | integer                  | 
 length_meters      | double precision         | 
 added_by_id        | integer                  | 
 date_created       | timestamp with time zone | 
 last_modified      | timestamp with time zone | 
 normalized_name    | character varying(64)    | not null
 rating_Votes       | integer                  | not null
 rating_score       | integer                  | not null
Indexes:
    "web_route_pkey" PRIMARY KEY,btree (id)
    "web_route_added_by_id" btree (added_by_id)
    "web_route_crag_id" btree (crag_id)
Check constraints:
    "ck_rating_Votes_pstv_c39bae29f3b2012" CHECK (rating_Votes >= 0)
    "web_route_rating_Votes_check" CHECK (rating_Votes >= 0)
Foreign-key constraints:
    "added_by_id_refs_id_157791930f5e12d5" FOREIGN KEY (added_by_id) REFERENCES web_member(id) DEFERRABLE INITIALLY DEFERRED
    "crag_id_refs_id_3ce1145606d12740" FOREIGN KEY (crag_id) REFERENCES web_crag(id) DEFERRABLE INITIALLY DEFERRED

解决方法

遗憾的是,Postgresql还不擅长优化这些类型的排序,如果它找不到与sort子句完全匹配的索引,它总是希望立即对整个结果集进行排序.

从Postgresql 9.3开始,你可以欺骗规划者用LATERAL subquery做正确的事情.试试这个：

SELECT "web_route"."id","web_crag"."id"
FROM "web_crag",LAteraL (
    SELECT * FROM "web_route"
    WHERE "web_route"."crag_id" = "web_crag"."id"
    ORDER BY "web_route"."id" ASC
) AS "web_route"
WHERE "web_crag"."type" IN (1,2)
ORDER BY "web_crag"."name"
LIMIT 20;

我生成了一些简单的测试数据(100万web_crags,500万web_routes),这里是查询计划和时间……除了额外的web_route.id排序外,几乎与您的第一个查询计划完全相同：

Limit  (cost=24.36..120.70 rows=20 width=14) (actual time=0.051..0.169 rows=20 loops=1)
   ->  nested Loop  (cost=24.36..24084788.95 rows=5000000 width=14) (actual time=0.049..0.143 rows=20 loops=1)
         ->  Index Scan using web_crag_name_idx on web_crag  (cost=0.42..39131.46 rows=1000000 width=10) (actual time=0.018..0.023 rows=4 loops=1)
               Filter: (type = ANY ('{1,2}'::integer[]))
         ->  Sort  (cost=23.93..23.95 rows=5 width=8) (actual time=0.018..0.021 rows=5 loops=4)
               Sort Key: web_route.id
               Sort Method: quicksort  Memory: 25kB
               ->  Index Scan using web_route_crag_id_idx on web_route  (cost=0.43..23.88 rows=5 width=8) (actual time=0.005..0.011 rows=5 loops=4)
                     Index Cond: (crag_id = web_crag.id)
 Total runtime: 0.212 ms

您可以使用web_route(crag_id,id)上的附加索引来避免排序：

Limit  (cost=0.86..19.49 rows=20 width=14) (actual time=0.031..0.113 rows=20 loops=1)
   ->  nested Loop  (cost=0.86..4659293.82 rows=5000000 width=14) (actual time=0.029..0.084 rows=20 loops=1)
         ->  Index Scan using web_crag_name_idx on web_crag  (cost=0.42..39293.82 rows=1000000 width=10) (actual time=0.017..0.021 rows=4 loops=1)
               Filter: (type = ANY ('{1,2}'::integer[]))
         ->  Index Only Scan using web_route_crag_id_id_idx on web_route  (cost=0.43..4.52 rows=5 width=8) (actual time=0.005..0.009 rows=5 loops=4)
               Index Cond: (crag_id = web_crag.id)
               Heap Fetches: 0
 Total runtime: 0.151 ms

这是我创建测试数据的方式：

create table web_crag(id serial primary key,type int default 1,name text);
create table web_route(id serial primary key,crag_id int);
insert into web_crag (name) select generate_series(1,1000000)::text;
insert into web_route (crag_id) select id from web_crag cross join generate_series(1,5);
create index on web_crag(name);
create index on web_route(crag_id);
analyze web_route;

Postgresql补丁

有一个“partial sort” patch to PostgreSQL可以自动进行大致这种优化,但遗憾的是它没有为Postgresql 9.4做出决定.希望Postgresql 9.5能够拥有它(大约在2015年下半年).

版权声明：本文内容由互联网用户自发贡献，该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容， 请发送邮件至 dio@foxmail.com 举报，一经查实，本站将立刻删除。

相关推荐

PostGreSQL到Oracle的数据迁移
项目需要，有个数据需要导入，拿到手一开始以为是mysql，结果是个PostGreSQL的数据，于是装数据库，但这个也不懂呀，而且本系统用的Oracle，于是得解决迁移转换的问题。 总结下来两个思路。1、PostgresToOracle多金的
怎么查看PostgreSQL数据库中所有表
本文小编为大家详细介绍“怎么查看PostgreSQL数据库中所有表”，内容详细，步骤清晰，细节处理妥当，希望这篇“怎么查看PostgreSQL数据库中所有表”文章能帮助大...
postgresql 报错 FATAL: no pg_hba.conf entry for host 未配置允许远程连接 解决方法
错误现象问题原因这是在远程连接时pg_hba.conf文件没有配置正确。  pg_hba.conf文件在Postgre安装文件目录下的data文件夹中。解决方案1、进入到data目录下，找到pg_hba.conf文件
postgresql 数据库 报错 server closed the connection unexpectedly 无法远程连接数据库 解决方法
因本地资源有限，在公共测试环境搭建了PGsql环境，从数据库本地localhost访问正常，在相同网段的远程机器访问报如下错误
postgresql 报错 SQLSTATE[08006] [7] SCRAM authentication requires libpq version 10 or above 解决方法
wamp 环境 这个提示就是说你的版本低于10了。 先打印php_info(),查看自己的版本（我这边是已经处理后的，之前的忘记截图了）
postgresql 报错 psycopg2.OperationalError: SSL SYSCALL error: EOF detected 解决方法
psycopg2.OperationalError: SSL SYSCALL error: EOF detected 问题提示:exception psycopg2.OperationalError
postgresql 数据库报错 FATAL: sorry, too many clients already 解决方法
项目 postgres 连接不上， 所有连接报错 ：psql: FATAL: sorry, too many clients already问题原由程序使用连接未及时释放， 连接一直处于 idle 状态处理方式1、 程序里面未释放的连接， 在使用后及时释放
postgresql 报错 FATAL: the database system is shutting down 解决方法
服务器在同一个数据目录上启动了两个PostgreSQL实例(它已经删除postmaster.pid并使用了新的端口号,因此绕过了这种行为的正常保护措施被绕过),导致PostgreSQL的误操作postgresql 报错 FATAL: the database system is
postgresql报错 FATAL: the database system is in recovery mode 解决方法
问题原因：数据库崩溃，内存不足造成 或者 数据已损坏，磁盘故障造成首先介绍一下背景，在测试Deepgreen（Greenplum升级版）数据库时，pgbench并发数设置过多，导致数据库卡死了，在进行连接、重启、关闭时，都报同样
2021 PostgreSQL 中国技术大会 PPT 下载
第 11 届 PostgreSQL 中国技术大会于 2022 年 1 月 7 日至 9 日在武汉光谷会展酒店成功举办。作为 PostgreSQL 技术领域的年度盛事，postgreSQL 中文社区旨在搭建开放、合作共享的平台，基于开源，创新驱动，共同探讨数据库行业数字化发展方向和未来新机遇。
苹果市值2025年有望达4万亿美元
pythonJavaScriptjavaHTMLreactjsC#AndroidCSSNode.jssqlrpython-3.xMysqLjQueryc++pandasFlutterangularIOSdjangolinuxswifttypescript路由器JSON路由器设置无线路由器h3c华三华三路由器设置华三路由器电脑软件教程arraysdocker软件图文教程Cvue.jslaravelspring-bootreact-native