我正在尝试对我们需要为内部目的运行的查询进行故障排除。此查询不是应用程序的一部分,因此它不必那么高效,但我希望它尽可能快,以便我们可以获得前进所需的数据集。
我遇到的第一个问题是查询卡在统计状态。通过将以下内容添加到我们的 mySQL 配置中来解决这个问题......
optimizer_search_depth = 0
现在查询直接进入“将数据复制到 tmp 表”,到目前为止已经发生了 5 多分钟。我们的服务器非常强大,我们有 128GB 的 RAM,所以我认为这根本不是资源问题。我确实相信查询本身可能需要优化一点,如果可能的话,这就是我想要的指导。
“EXPLAIN SELECT”的结果can be seen here .在运行 EXPLAIN SELECT 之前,我对该数据库中的所有表(为一次性查询而创建)执行了“ANALYZE”。
查询本身在这里:
SELECT DISTINCT
IAN.sku,
IAN.notes,
Parts.partterminologyname,
BaseVehicle.YearID,
Make.MakeName,
Model.modelname,
SubModel.SubModelName,
CONCAT(EngineBase.Cylinders, ' Cyl ', EngineBase.Liter, 'L') as engine,
Positions.position,
BedLength.BedLength,
BedLength.BedLengthMetric,
BedType.BedTypeName,
BodyNumDoors.BodyNumDoors,
BodyType.BodyTypeName,
FrontBrakeType.BrakeTypeName,
RearBrakeType.BrakeTypeName,
BrakeSystem.BrakeSystemName,
BrakeABS.BrakeABSName,
DriveType.DriveTypeName,
EngineDesignation.EngineDesignationName,
EngineVIN.EngineVINName,
Valves.ValvesPerEngine,
EngineBase.Liter,
EngineBase.CC,
EngineBase.CID,
EngineBase.Cylinders,
EngineBase.BlockType,
EngineBase.EngBoreIn,
EngineBase.EngBoreMetric,
EngineBase.EngStrokeIn,
EngineBase.EngStrokeMetric,
FuelDeliveryType.FuelDeliveryTypeName,
FuelDeliverySubType.FuelDeliverySubTypeName,
FuelSystemControlType.FuelSystemControlTypeName,
FuelSystemDesign.FuelSystemDesignName,
Aspiration.AspirationName,
CylinderHeadType.CylinderHeadTypeName,
FuelType.FuelTypeName,
IgnitionSystemType.IgnitionSystemTypeName,
EngineMfr.MfrName,
EngineVersion.EngineVersion,
PowerOutput.HorsePower,
PowerOutput.KilowattPower,
MfrBodyCode.MfrBodyCodeName,
SpringType.SpringTypeName,
SteeringType.SteeringTypeName,
SteeringSystem.SteeringSystemName,
TransmissionType.TransmissionTypeName,
TransmissionNumSpeeds.TransmissionNumSpeeds,
TransmissionMfrCode.TransmissionMfrCode,
TransElecControlled.ElecControlled,
TransmissionMfr.MfrName,
WheelBase.WheelBase,
WheelBase.WheelBaseMetric
FROM
Import_AcesApplication_New IAN
INNER JOIN BaseVehicle
ON IAN.base_vehicle_id = BaseVehicle.BaseVehicleID
INNER JOIN Make
ON BaseVehicle.MakeID = Make.MakeID
INNER JOIN Model
ON BaseVehicle.ModelID = Model.ModelID
INNER JOIN Positions
ON IAN.position_id = Positions.PositionID
INNER JOIN Parts
ON IAN.part_type_id = Parts.PartTerminologyID
INNER JOIN Vehicle
ON IAN.base_vehicle_id = Vehicle.BaseVehicleID
INNER JOIN SubModel
ON Vehicle.SubModelID = SubModel.SubModelID
INNER JOIN VehicleConfig
ON Vehicle.VehicleID = VehicleConfig.VehicleID
INNER JOIN EngineConfig
ON VehicleConfig.EngineConfigID = EngineConfig.EngineConfigID
INNER JOIN EngineBase
ON EngineConfig.EngineBaseID = EngineBase.EngineBaseID
INNER JOIN EngineDesignation
ON EngineConfig.EngineDesignationID = EngineDesignation.EngineDesignationID
INNER JOIN EngineVIN
ON EngineConfig.EngineVINID = EngineVIN.EngineVINID
INNER JOIN Valves
ON EngineConfig.ValvesID = Valves.Valvesid
INNER JOIN FuelDeliveryConfig
ON EngineConfig.FuelDeliveryConfigID = FuelDeliveryConfig.FuelDeliveryConfigID
INNER JOIN FuelDeliveryType
ON FuelDeliveryConfig.FuelDeliveryTypeID = FuelDeliveryType.FuelDeliveryTypeID
INNER JOIN FuelDeliverySubType
ON FuelDeliveryConfig.FuelDeliverySubTypeID = FuelDeliverySubType.FuelDeliverySubTypeID
INNER JOIN FuelSystemControlType
ON FuelDeliveryConfig.FuelSystemControlTypeID = FuelSystemControlType.FuelSystemControlTypeID
INNER JOIN FuelSystemDesign
ON FuelDeliveryConfig.FuelSystemDesignID = FuelSystemDesign.FuelSystemDesignID
INNER JOIN Aspiration
ON EngineConfig.AspirationID = Aspiration.AspirationID
INNER JOIN CylinderHeadType
ON EngineConfig.CylinderHeadTypeID = CylinderHeadType.CylinderHeadTypeID
INNER JOIN FuelType
ON EngineConfig.FuelTypeID = FuelType.FuelTypeID
INNER JOIN IgnitionSystemType
ON EngineConfig.IgnitionSystemTypeID = IgnitionSystemType.IgnitionSystemTypeID
INNER JOIN Mfr EngineMfr
ON EngineConfig.EngineMfrID = EngineMfr.MfrID
INNER JOIN EngineVersion
ON EngineConfig.EngineVersionID = EngineVersion.EngineVersionID
INNER JOIN PowerOutput
ON EngineConfig.PowerOutputId = PowerOutput.PowerOutputId
INNER JOIN BedConfig
ON VehicleConfig.BedConfigID = BedConfig.BedConfigID
INNER JOIN BedLength
ON BedConfig.BedLengthID = BedLength.BedLengthID
INNER JOIN BedType
ON BedConfig.BedTypeID = BedType.BedTypeID
INNER JOIN BodyStyleConfig
ON VehicleConfig.BodyStyleConfigID = BodyStyleConfig.BodyStyleConfigID
INNER JOIN BodyNumDoors
ON BodyStyleConfig.BodyNumDoorsID = BodyNumDoors.BodyNumDoorsID
INNER JOIN BodyType
ON BodyStyleConfig.BodyTypeID = BodyType.BodyTypeID
INNER JOIN BrakeConfig
ON VehicleConfig.BrakeConfigID = BrakeConfig.BrakeConfigID
INNER JOIN BrakeType FrontBrakeType
ON BrakeConfig.FrontBrakeTypeID = FrontBrakeType.BrakeTypeID
INNER JOIN BrakeType RearBrakeType
ON BrakeConfig.RearBrakeTypeID = RearBrakeType.BrakeTypeID
INNER JOIN BrakeSystem
ON BrakeConfig.BrakeSystemID = BrakeSystem.BrakeSystemID
INNER JOIN BrakeABS
ON BrakeConfig.BrakeABSID = BrakeABS.BrakeABSID
INNER JOIN DriveType
ON VehicleConfig.DriveTypeID = DriveType.DriveTypeID
INNER JOIN MfrBodyCode
ON VehicleConfig.MfrBodyCodeID = MfrBodyCode.MfrBodyCodeID
INNER JOIN SpringType
ON VehicleConfig.SpringTypeConfigID = SpringType.SpringTypeID
INNER JOIN SteeringConfig
ON VehicleConfig.SteeringConfigID = SteeringConfig.SteeringConfigID
INNER JOIN SteeringType
ON SteeringConfig.SteeringConfigID = SteeringType.SteeringTypeID
INNER JOIN SteeringSystem
ON SteeringConfig.SteeringSystemID = SteeringSystem.SteeringSystemID
INNER JOIN Transmission
ON VehicleConfig.TransmissionID = Transmission.TransmissionID
INNER JOIN TransmissionBase
ON Transmission.TransmissionBaseID = TransmissionBase.TransmissionBaseID
INNER JOIN TransmissionType
ON TransmissionBase.TransmissionTypeID = TransmissionType.TransmissionTypeID
INNER JOIN TransmissionNumSpeeds
ON TransmissionBase.TransmissionNumSpeedsID = TransmissionNumSpeeds.TransmissionNumSpeedsID
INNER JOIN TransmissionControlType
ON TransmissionBase.TransmissionControlTypeID = TransmissionControlType.TransmissionControlTypeID
INNER JOIN TransmissionMfrCode
ON Transmission.TransmissionMfrCodeID = TransmissionMfrCode.TransmissionMfrCodeID
INNER JOIN ElecControlled TransElecControlled
ON Transmission.TransmissionElecControlledID = TransElecControlled.ElecControlledID
INNER JOIN Mfr TransmissionMfr
ON Transmission.TransmissionMfrID = TransmissionMfr.MfrID
INNER JOIN WheelBase
ON VehicleConfig.WheelbaseID = WheelBase.WheelBaseID
LIMIT
0,10
由于数据库的设置方式,查询中有许多主要的 JOINS。数据库是我们行业的标准,我不想改变它。应该注意的是,这个查询可能只会每隔几个月运行一次,因此虽然优化它很重要,但它不一定非常高效。对最终结果的唯一期望是我们不会日复一日地等待数据准备就绪。
我很感激任何关于这方面的指导,如果这有点需要征求意见,我深表歉意。
最佳答案
您的查询看起来布局完美。您的主表是“Import_AcesApplication_New”表,其余所有表都是从主表派生的查找表。我有一个类似的问题,政府拨款/契约(Contract)查询查找 20 多个表并且查询被阻塞,这是针对 15+ 百万条记录
只需添加关键字
“STRAIGHT_JOIN”
选择 STRAIGHT_JOIN [其余查询]
它强制引擎“按照您提供的顺序”运行查询。它可能会尝试过度分析自己,了解哪个表具有最少的记录、最佳索引等,并通过一些奇怪的表选择向后工作。
此外,编辑原始查询以更好地通过层次结构显示关联链接以提高可读性
我希望您所有的查找表都在其主键上有索引,但您也可以通过覆盖索引受益,这样每次查找的原始数据页都不是必需的。
Table Index suggestion
Import_AcesApplication_New ( sku, notes, base_vehicle_id, position_id, part_type_id )
BaseVehicle ( BaseVehicleID, MakeID, ModelID, YearID )
Make ( MakeID, MakeName )
Model ( ModelID, modelname )
Positions ( PositionID, Position )
Parts ( PartTerminologyID, partterminologyname )
Vehicle ( BaseVehicleID, SubModelID, VehicleID )
SubModel ( SubModelID, SubModelName )
VehicleConfig ( VehicleID ) -- too many other fields to be practical
EngineConfig ( EngineConfigID ) -- too many to be practical
EngineBase ( EngineBaseID ) -- too many to be practical
EngineDesignation ( EngineDesignationID, EngineDesignationName )
EngineVIN ( EngineVINID, EngineVINName )
Valves ( Valvesid, ValvesPerEngine )
FuelDeliveryConfig ( FuelDeliveryConfigID, FuelDeliveryTypeID, FuelDeliverySubTypeID, FuelSystemControlTypeID, FuelSystemDesignID )
FuelDeliveryType ( FuelDeliveryTypeID, FuelDeliveryTypeName )
FuelDeliverySubType ( FuelDeliverySubTypeID, FuelDeliverySubTypeName )
FuelSystemControlType ( FuelSystemControlTypeID, FuelSystemControlTypeName )
FuelSystemDesign ( FuelSystemDesignID, FuelSystemDesignName )
Aspiration ( AspirationID, AspirationName )
CylinderHeadType ( CylinderHeadTypeID, CylinderHeadTypeName )
FuelType ( FuelTypeID, FuelTypeName )
IgnitionSystemType ( IgnitionSystemTypeID, IgnitionSystemTypeName )
EngineMfr ( MfrID, MfrName )
EngineVersion ( EngineVersionID, EngineVersion )
PowerOutput ( PowerOutputId, HorsePower, KilowattPower )
BedConfig ( BedConfigID, BedLengthID, BedTypeID )
BedLength ( BedLengthID, BedLength, BedLengthMetric
BedType ( BedTypeID, BedTypeName )
BodyStyleConfig ( BodyStyleConfigID, BodyNumDoorsID, BodyTypeID )
BodyNumDoors ( BodyNumDoorsID, BodyNumDoors )
BodyType ( BodyTypeID, BodyTypeName )
BrakeConfig ( BrakeConfigID, FrontBrakeTypeID, RearBrakeTypeID, BrakeSystemID, BrakeABSID )
BrakeType ( BrakeTypeID, BrakeTypeName )
BrakeSystem ( BrakeSystemID, BrakeSystemName )
BrakeABS ( BrakeABSID, BrakeABSName )
DriveType ( DriveTypeID, DriveTypeName )
MfrBodyCode ( MfrBodyCodeID, MfrBodyCodeName )
SpringType ( SpringTypeID, SpringTypeName )
SteeringConfig ( SteeringConfigID, SteeringTypeID, SteeringSystemID) -- see note below indexes via JOIN
SteeringType ( SteeringTypeID, SteeringTypeName )
SteeringSystem ( SteeringSystemID, SteeringSystemName )
Transmission ( TransmissionID, TransmissionBaseID )
TransmissionBase ( TransmissionBaseID, TransmissionTypeID, TransmissionNumSpeedsID, TransmissionControlTypeID )
TransmissionType ( TransmissionTypeID, TransmissionTypeName )
TransmissionNumSpeeds ( TransmissionNumSpeedsID, TransmissionNumSpeeds )
TransmissionControlType ( TransmissionControlTypeID, ??? ) -- possible description column per note below
TransmissionMfrCode ( TransmissionMfrCodeID, TransmissionMfrCode )
ElecControlled ( ElecControlledID, ElecControlled )
Mfr ( MfrID, MfrName )
WheelBase ( WheelBaseID, WheelBase, WheelBaseMetric )
此外,在查看 JOIN 以获得覆盖索引时,我注意到转向配置与转向类型的连接是基于
ON SteeringConfig.SteeringConfigID = SteeringType.SteeringTypeID
这应该是……
ON SteeringConfig.SteeringTypeID = SteeringType.SteeringTypeID
在另一个项目上... TransmissionControlType 已连接到,但您没有从输出查询中提取任何列...可能是对这么多表/连接的轻微疏忽。
这些应该对查询中的覆盖索引有很大帮助
关于mysql - 使用许多连接优化 SQL 查询,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/22046796/
我正在学习如何使用Nokogiri,根据这段代码我遇到了一些问题:require'rubygems'require'mechanize'post_agent=WWW::Mechanize.newpost_page=post_agent.get('http://www.vbulletin.org/forum/showthread.php?t=230708')puts"\nabsolutepathwithtbodygivesnil"putspost_page.parser.xpath('/html/body/div/div/div/div/div/table/tbody/tr/td/div
我有一个Ruby程序,它使用rubyzip压缩XML文件的目录树。gem。我的问题是文件开始变得很重,我想提高压缩级别,因为压缩时间不是问题。我在rubyzipdocumentation中找不到一种为创建的ZIP文件指定压缩级别的方法。有人知道如何更改此设置吗?是否有另一个允许指定压缩级别的Ruby库? 最佳答案 这是我通过查看rubyzip内部创建的代码。level=Zlib::BEST_COMPRESSIONZip::ZipOutputStream.open(zip_file)do|zip|Dir.glob("**/*")d
类classAprivatedeffooputs:fooendpublicdefbarputs:barendprivatedefzimputs:zimendprotecteddefdibputs:dibendendA的实例a=A.new测试a.foorescueputs:faila.barrescueputs:faila.zimrescueputs:faila.dibrescueputs:faila.gazrescueputs:fail测试输出failbarfailfailfail.发送测试[:foo,:bar,:zim,:dib,:gaz].each{|m|a.send(m)resc
很好奇,就使用rubyonrails自动化单元测试而言,你们正在做什么?您是否创建了一个脚本来在cron中运行rake作业并将结果邮寄给您?git中的预提交Hook?只是手动调用?我完全理解测试,但想知道在错误发生之前捕获错误的最佳实践是什么。让我们理所当然地认为测试本身是完美无缺的,并且可以正常工作。下一步是什么以确保他们在正确的时间将可能有害的结果传达给您? 最佳答案 不确定您到底想听什么,但是有几个级别的自动代码库控制:在处理某项功能时,您可以使用类似autotest的内容获得关于哪些有效,哪些无效的即时反馈。要确保您的提
假设我做了一个模块如下:m=Module.newdoclassCendend三个问题:除了对m的引用之外,还有什么方法可以访问C和m中的其他内容?我可以在创建匿名模块后为其命名吗(就像我输入“module...”一样)?如何在使用完匿名模块后将其删除,使其定义的常量不再存在? 最佳答案 三个答案:是的,使用ObjectSpace.此代码使c引用你的类(class)C不引用m:c=nilObjectSpace.each_object{|obj|c=objif(Class===objandobj.name=~/::C$/)}当然这取决于
我正在尝试使用ruby和Savon来使用网络服务。测试服务为http://www.webservicex.net/WS/WSDetails.aspx?WSID=9&CATID=2require'rubygems'require'savon'client=Savon::Client.new"http://www.webservicex.net/stockquote.asmx?WSDL"client.get_quotedo|soap|soap.body={:symbol=>"AAPL"}end返回SOAP异常。检查soap信封,在我看来soap请求没有正确的命名空间。任何人都可以建议我
关闭。这个问题是opinion-based.它目前不接受答案。想要改进这个问题?更新问题,以便editingthispost可以用事实和引用来回答它.关闭4年前。Improvethisquestion我想在固定时间创建一系列低音和高音调的哔哔声。例如:在150毫秒时发出高音调的蜂鸣声在151毫秒时发出低音调的蜂鸣声200毫秒时发出低音调的蜂鸣声250毫秒的高音调蜂鸣声有没有办法在Ruby或Python中做到这一点?我真的不在乎输出编码是什么(.wav、.mp3、.ogg等等),但我确实想创建一个输出文件。
我在我的项目目录中完成了compasscreate.和compassinitrails。几个问题:我已将我的.sass文件放在public/stylesheets中。这是放置它们的正确位置吗?当我运行compasswatch时,它不会自动编译这些.sass文件。我必须手动指定文件:compasswatchpublic/stylesheets/myfile.sass等。如何让它自动运行?文件ie.css、print.css和screen.css已放在stylesheets/compiled。如何在编译后不让它们重新出现的情况下删除它们?我自己编译的.sass文件编译成compiled/t
我正在用Ruby编写一个简单的程序来检查域列表是否被占用。基本上它循环遍历列表,并使用以下函数进行检查。require'rubygems'require'whois'defcheck_domain(domain)c=Whois::Client.newc.query("google.com").available?end程序不断出错(即使我在google.com中进行硬编码),并打印以下消息。鉴于该程序非常简单,我已经没有什么想法了-有什么建议吗?/Library/Ruby/Gems/1.8/gems/whois-2.0.2/lib/whois/server/adapters/base.
我想将html转换为纯文本。不过,我不想只删除标签,我想智能地保留尽可能多的格式。为插入换行符标签,检测段落并格式化它们等。输入非常简单,通常是格式良好的html(不是整个文档,只是一堆内容,通常没有anchor或图像)。我可以将几个正则表达式放在一起,让我达到80%,但我认为可能有一些现有的解决方案更智能。 最佳答案 首先,不要尝试为此使用正则表达式。很有可能你会想出一个脆弱/脆弱的解决方案,它会随着HTML的变化而崩溃,或者很难管理和维护。您可以使用Nokogiri快速解析HTML并提取文本:require'nokogiri'h