如何解决Ruby TinyTds gem 编码错误会阻止所有未来对数据库的访问
项目详情:
- Ruby 1.8.7
- tiny_tds gem 版本 0.6.0.rc1
- 连接到 microsoft sql 2017
- ActiveRecord 版本 2.3.14
我们遇到的问题是我们正尝试通过 rake 任务导出大约 150 万条记录。我遍历每条记录,然后将其导出到 json。一些数据已有 15 年的历史,一张表有一些奇怪的 sql_latin1_General_CP1_CI_AS 字符会破坏 tiny_tds。但最终会发生的事情是,如果我用错误的编码创下了记录(不是很多,在 150 万个中,我认为只有大约 9,000 个),那么此后的每一次迭代都会导致一个奇怪的 ActiveRecord/TinyTds 错误。
这是一个特定的列。我的解决方案是捕获错误,并将记录的 id 报告到文件中,代码如下:
association_names.to_a.each do |association_name|
puts "Exporting '#{association_name}' from '#{record.class} - #{record.id}'"
# associations = record.send(association_name).to_a
associations = begin
record.send(association_name).to_a
# rescue TinyTds::Error => e
rescue ActiveRecord::StatementInvalid => e
puts "HIT THE TINYTDS ERROR"
puts "#{e.class}: #{e.message}"
puts e.backtrace.join("\n")
File.open(File.join(pack_path,"overflow_errors.txt"),'a') do |f|
f.puts("Exporting '#{association_name}' from '#{record.class} - #{record.id}'")
end
[]
rescue StandardError => e
puts "HIT THE STANDARD ERROR"
puts "#{e.class}: #{e.message}"
puts e.backtrace.join("\n")
[]
end
associations.each do |association_record|
export_record(association_record,pack_path,scp)
end
end
它抛出错误的地方是record.send(association_name)
。第一个错误是 ActiveRecord::StatementInvalid: TinyTds::Error: Buffer overflow converting characters from client into server's character set
。我抓住,报告,然后跳到下一个。但是,当发生这种情况时,每个后续记录(没有编码问题的记录,我已经确认),我得到 ActiveRecord::StatementInvalid: NoMethodError: undefined method
each' for false:FalseClass`。我能够在控制台中重新创建错误,您可以在下面看到:
>> Service.find(33333).service_details
=> []
>> ServiceData.find(444444).service_data # Record with bad encoding
ActiveRecord::StatementInvalid: TinyTds::Error: Buffer overflow converting characters from client into server's character set
from /home/my_user/the_project/lib/rails/002_active_record/connection_adapters/abstract_adapter.rb:174:in `log'
from /home/my_user/the_project/lib/rails/002_active_record/connection_adapters/sqlserver_adapter.rb:657:in `raw_select'
from /home/my_user/the_project/lib/rails/002_active_record/connection_adapters/sqlserver_adapter.rb:632:in `select'
from /home/my_user/the_project/lib/rails/002_active_record/connection_adapters/abstract/database_statements.rb:7:in `select_all'
from /home/my_user/the_project/lib/rails/002_active_record/zbase.rb:489:in `find_by_sql'
from /home/my_user/the_project/lib/rails/002_active_record/zbase.rb:1360:in `find_every'
from /home/my_user/the_project/lib/rails/002_active_record/zbase.rb:1395:in `find_one'
from /home/my_user/the_project/lib/rails/002_active_record/zbase.rb:1381:in `find_from_ids'
from /home/my_user/the_project/lib/rails/002_active_record/zbase.rb:437:in `find'
from (irb):4
from :0
>> Service.find(33333).service_details
/home/my_user/the_project/lib/rails/002_active_record/connection_adapters/sqlserver_adapter.rb:659: warning: TinyTds: dbsqlsend() returned FAIL.
ActiveRecord::StatementInvalid: NoMethodError: undefined method `each' for false:FalseClass
from /home/my_user/the_project/lib/rails/002_active_record/connection_adapters/abstract_adapter.rb:174:in `log'
from /home/my_user/the_project/lib/rails/002_active_record/connection_adapters/sqlserver_adapter.rb:657:in `raw_select'
from /home/my_user/the_project/lib/rails/002_active_record/connection_adapters/sqlserver_adapter.rb:632:in `select'
from /home/my_user/the_project/lib/rails/002_active_record/connection_adapters/abstract/database_statements.rb:7:in `select_all'
from /home/my_user/the_project/lib/rails/002_active_record/zbase.rb:489:in `find_by_sql'
from /home/my_user/the_project/lib/rails/002_active_record/zbase.rb:1360:in `find_every'
from /home/my_user/the_project/lib/rails/002_active_record/zbase.rb:1395:in `find_one'
from /home/my_user/the_project/lib/rails/002_active_record/zbase.rb:1381:in `find_from_ids'
from /home/my_user/the_project/lib/rails/002_active_record/zbase.rb:437:in `find'
from (irb):5
from :0
我无法弄清楚为什么会发生这种情况。当 TinyTds::Error 发生时(由于某种原因只能用 ActiveRecord::StatementInvalid
捕获),它似乎锁定了数据库。同样,当我运行 rake 任务时,当发生后一个错误时,我也会收到此警告:warning: TinyTds: dbsqlsend() returned FAIL.
。
我已经深入研究了 activerecord 代码,它有时似乎确实返回 false,而不是一个空数组。我尝试更改它,但这导致了一系列其他问题。然而这个错误在生产中从未发生过,因为我们使用一个可以处理这种编码的 asp 服务器来提供这些数据。什么可能导致这种情况?谢谢。
版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 dio@foxmail.com 举报,一经查实,本站将立刻删除。