微信公众号搜"智元新知"关注
微信扫一扫可直接关注哦!

如何修复密码查询中的永恒执行 - Neo4j 图形数据库?

如何解决如何修复密码查询中的永恒执行 - Neo4j 图形数据库?

我正在使用密码语言查询和 apoc 库处理将常见弱点枚举目录(.json 文件)导入 Neo4j 图形数据库。尽管我正确导入了字段:Weaknesses、Views、External_References,但我在导入字段时遇到了执行问题(没有任何错误):Categories 正在执行而没有结束.下面我展示了 .json 文件的结构和我的密码。

"Weakness_Catalog": {
   "Weaknesses": {"Weakness":[...]}
   "Categories": {"Category":[...]}
   "Views": {"View":[...]}
   "External_References": {"External_Reference":[...]}
} 

密码查询

经过多次测试,我认为逻辑错误在最后两部分 [with value....(catRef)] 之间,没有它们,查询在正常时间执行得很好。我还更改了设置参数。由于错误 (cypher.lenient_create_relationship = true) 在 db 配置文件中。我测试了不同的导入序列,但结果相同(弱点、类别、视图、外部引用等)

call apoc.load.json(files) yield value
                                unwind value.Weakness_Catalog.Weaknesses.Weakness as weakness
                                merge (i:GeneralInfo_CWE {Name:value.Weakness_Catalog.Name,Version:value.Weakness_Catalog.Version,Date:value.Weakness_Catalog.Date,Schema:'https://cwe.mitre.org/data/xsd/cwe_schema_v6.4.xsd'})
                                
                                merge(w:CWE {Name:'CWE-' + weakness.ID})
                                set w.Extended_Name=weakness.Name,w.Abstraction=weakness.Abstraction,w.Structure=weakness.Structure,w.Status=weakness.Status,w.Description=weakness.Description,w.Extended_Description= apoc.convert.toString(weakness.Extended_Description),w.Likelihood_Of_Exploit=weakness.Likelihood_Of_Exploit,w.Background_Details=apoc.convert.toString(weakness.Background_Details.Background_Detail),w.Modes_Of_Introduction=[value in weakness.Modes_Of_Introduction.Introduction | value.Phase],w.Submission_Date=weakness.Content_History.Submission.Submission_Date,w.Submission_Name=weakness.Content_History.Submission.Submission_Name,w.Submission_Organization=weakness.Content_History.Submission.Submission_Organization,w.Modifications=[value in weakness.Content_History.Modification | apoc.convert.toString(value)],w.Alternate_Terms=apoc.convert.toString(weakness.Alternate_Terms),w.Notes=[value in weakness.Notes.Note | apoc.convert.toString(value)],w.Affected_Resources=[value in weakness.Affected_Resources.Affected_Resource | value],w.Functional_Areas=[value in weakness.Functional_Areas.Functional_Area | value]
                                
                                merge (w)-[:belongsTo]->(i)
                                
                                with w,weakness,value
                                unwind weakness.Related_Weaknesses.Related_Weakness as Rel_Weakness
                                match (cwe:CWE) where cwe.Name='CWE-' + Rel_Weakness.CWE_ID
                                merge (w)-[:Related_Weakness{Nature:Rel_Weakness.Nature}]->(cwe)
                                
                                with w,value
                                unwind weakness.Applicable_Platforms as appPl
                                    foreach (lg in appPl.Language |
                                        merge(ap:Applicable_Platform{Type:'Language',Prevalence:lg.Prevalence,Name:coalesce(lg.Name,'NOT SET'),Class:coalesce(lg.Class,'NOT SET')})
                                        merge(w)-[:Applicable_Platform]->(ap))
                                
                                    with w,value,appPl
                                    foreach (tch in appPl.Technology |
                                        merge(ap:Applicable_Platform{Type:'Technology',Prevalence:tch.Prevalence,Name:coalesce(tch.Name,Class:coalesce(tch.Class,'NOT SET')})
                                        merge(w)-[:Applicable_Platform]->(ap))
                                   
                                    with w,appPl
                                    foreach (arc in appPl.Architecture |
                                        merge(ap:Applicable_Platform{Type:'Architecture',Prevalence:arc.Prevalence,Name:coalesce(arc.Name,Class:coalesce(arc.Class,'NOT SET')})
                                        merge(w)-[:Applicable_Platform]->(ap))
                                    
                                    with w,appPl
                                    foreach (os in appPl.Operating_System |
                                        merge(ap:Applicable_Platform{Type:'Operating System',Prevalence:os.Prevalence,Name:coalesce(os.Name,Class:coalesce(os.Class,'NOT SET')})
                                        merge(w)-[:Applicable_Platform]->(ap))             
                           
                           
                                with w,value
                                foreach (example in weakness.Demonstrative_Examples.Demonstrative_Example |
                                    merge(ex:Demonstrative_Example {Intro_Text:apoc.convert.toString(example.Intro_Text)})
                                    set ex.Body_Text=[value in example.Body_Text | apoc.convert.toString(value)],ex.Example_Code=[value in example.Example_Code | apoc.convert.toString(value)]                                
                                    merge (w)-[:hasExample]->(ex))
                                                                
                                                     
                                with w,value
                                foreach (consequence in weakness.Common_Consequences.Consequence |
                                    merge (con:Consequence{CWE:w.Name,Scope:[value in consequence.Scope | value]})
                                    set con.Impact=[value in consequence.Impact | value],con.Note=consequence.Note,con.Likelihood=consequence.Likelihood
                                    merge(w)-[:hasConsequence]->(con))
                                
                                with w,value
                                foreach (dec in weakness.Detection_Methods.Detection_Method |
                                    merge(d:Detection_Method {Method:dec.Method})
                                    merge(w)-[wd:canbedetected{Description:apoc.convert.toString(dec.Description)}]->(d)
                                    set wd.Effectiveness=dec.Effectiveness,wd.Effectiveness_Notes=dec.Effectiveness_Notes,wd.Detection_Method_ID=dec.Detection_Method_ID)                                
                                
                                with w,value
                                foreach (mit in weakness.Potential_Mitigations.Mitigation |
                                    merge(m:Mitigation {Description:apoc.convert.toString(mit.Description)})
                                    set m.Phase=[value in mit.Phase | value],m.Strategy=mit.Strategy,m.Effectiveness=mit.Effectiveness,m.Effectiveness_Notes=mit.Effectiveness_Notes,m.Mitigation_ID=mit.Mitigation_ID
                                    merge(w)-[:hasMitigation]->(m))
                                    
                                with w,value
                                foreach (rap in weakness.Related_Attack_Patterns.Related_Attack_Pattern |
                                    merge(cp:CAPEC {Name:rap.CAPEC_ID})
                                    merge(w)-[:RelatedAttackPattern]->(cp))
                                
                                with w,value
                                foreach (reference in value.Weakness_Catalog.External_References.External_Reference |
                                    merge(r:External_Reference{Reference_ID:reference.Reference_ID})
                                    set r.Author=[value in reference.Author | value],r.Title=reference.Title,r.Edition=reference.Edition,r.URL=reference.URL,r.Publication_Year=reference.Publication_Year,r.Publisher=reference.Publisher)
                                
                                with w,value
                                unwind weakness.References.Reference as exReference
                                match (ref:External_Reference) where ref.Reference_ID=exReference.External_Reference_ID
                                merge(w)-[:hasExternal_Reference]->(ref)
                                                 
                                                                
                                with value
                                unwind value.Weakness_Catalog.Views.View as view
                                merge (v:CWE_VIEW{ViewID:view.ID})
                                set v.Name=view.Name,v.Type=view.Type,v.Status=view.Status,v.Objective=apoc.convert.toString(view.Objective),v.Filter=view.Filter,v.Notes=apoc.convert.toString(view.Notes),v.Submission_Name=view.Content_History.Submission.Submission_Name,v.Submission_Date=view.Content_History.Submission.Submission_Date,v.Submission_Organization=view.Content_History.Submission.Submission_Organization,v.Modification=[value in view.Content_History.Modification | apoc.convert.toString(value)]
                                
                                foreach (value in view.Audience.Stakeholder |
                                    merge (st:Stakeholder{Type:value.Type})
                                    merge (v)-[rel:usefulFor]->(st)
                                    set rel.Description=value.Description)
                                
                                with v,view,value
                                unwind (case view.Members.Has_Member when [] then [null] else view.Members.Has_Member end) as members
                                optional match (MemberWeak:CWE{Name:'CWE-' + members.CWE_ID})
                                merge (v)-[:hasMember{ViewID:members.View_ID}]->(MemberWeak)                                
                                
                                                                                                                                
                                with v,value
                                unwind (case view.References.Reference when [] then [null] else view.References.Reference end) as viewExReference
                                optional match (viewRef:External_Reference{Reference_ID:viewExReference.External_Reference_ID})
                                merge (v)-[:hasExternal_Reference{ViewID:v.ViewID}]->(viewRef)
                                
                                with value
                                unwind value.Weakness_Catalog.Categories.Category as category
                                merge (c:CWE_Category{CategoryID:category.ID})
                                set c.Name=category.Name,c.Status=category.Status,c.Summary=apoc.convert.toString(category.Summary),c.Notes=apoc.convert.toString(category.Notes),c.Submission_Name=category.Content_History.Submission.Submission_Name,c.Submission_Date=category.Content_History.Submission.Submission_Date,c.Submission_Organization=category.Content_History.Submission.Submission_Organization,c.Modification=[value in category.Content_History.Modification | apoc.convert.toString(value)]
                                
                                with c,category
                                unwind (case category.References.Reference when [] then [null] else category.References.Reference end) as categoryExReference
                                optional match (catRef:External_Reference{Reference_ID:categoryExReference.External_Reference_ID})
                                merge (c)-[:hasExternal_Reference{CategoryID:c.CategoryID}]->(catRef)

解决方法

所以,问题是每次我使用 with 时,我都在嵌套循环中工作。嵌套循环越多,查询越慢。加快速度的一个好方法是在可能的情况下创建更简单的查询。

例如在json文件中:

"Weakness_Catalog": {
   "Weaknesses": {"Weakness":[...]}
   "Categories": {"Category":[...]}
   "Views": {"View":[...]}
   "External_References": {"External_Reference":[...]}
} 

我将针对弱点执行一项查询,一项针对类别,一项针对视图,一项针对 External_References。

版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 dio@foxmail.com 举报,一经查实,本站将立刻删除。