如何解决如何从我试图在字符串中抓取的网页中获取 html?
我编写了以下代码:
require "http/client"
require "myhtml"
puts "Give me the URL of the page to be scraped."
url = gets
html=<<-HTML
[Here goes the html of the website to be scraped]
HTML
myhtml = Myhtml::Parser.new(html)
myhtml.nodes(:div).each do |node|
id = node.attribute_by("id")
if first_link = node.scope.nodes(:a).first?
href = first_link.attribute_by("href")
link_text = first_link.inner_text
puts "div with id #{id} have link [#{link_text}](#{href})"
else
puts "div with id #{id} have no links"
end
end
html=<<-HTML
[Here goes the html of the website to be scraped]
HTML
类似的东西
response = requests.get(url)
html = BeautifulSoup(response.text,'html.parser')
来自以下 Python 代码:
url = input("What is the address of the web page in question?\n")
response = requests.get(url)
html = BeautifulSoup(response.text,'html.parser')
或 let html = reqwest::get(url).await?.text().await?;
来自以下 Rust 代码:
println!("Give me the URL of the page to be scraped.");
let mut url = String::new();
io::stdin().read_line(&mut url).expect("Failed to read line");
let html = reqwest::get(url).await?.text().await?;
shard myhtml 的文档没有提供足够的 例子让我弄清楚这一点。 可以用 Crystal's HTTP client 从他们的 标准库? 当我更换
html=<<-HTML
[Here goes the html of the website to be scraped]
HTML
与
response = HTTP::Client.get url
html = response.body
我收到以下错误:
response = HTTP::Client.get url #no overload matches 'HTTP::Client.get' with type (String | Nil)
^--
Error: no overload matches 'HTTP::Client.get' with type (String | Nil)
Overloads are:
- HTTP::Client.get(url : String | URI,headers : HTTP::Headers | ::Nil = nil,body : BodyType = nil,tls : TLSContext = nil)
- HTTP::Client.get(url : String | URI,tls : TLSContext = nil,&block)
- HTTP::Client.get(url,*,form : String | IO | Hash)
- HTTP::Client.get(url,form : String | IO | Hash,&block)
Couldn't find overloads for these types:
- HTTP::Client.get(Nil)
我能够从网页中获取文本
通过硬编码,例如response = HTTP::Client.get "https://github.com/monero-project/monero/releases"
但这还不够,因为我希望应用程序具有交互性。
解决方法
你已经接近了,抱怨的是类型系统。 HTTP::Client.get
需要 String
(或者更确切地说是 String | URL
)。但是,在您的代码中,您的 url
变量也可以是 nil
并且属于 String?
类型,它是 String | Nil
的缩写。如果您对 URL 进行硬编码,则它不能是 nil
,而始终是 String
类型。因此 HTTP::Client.get
调用有效。
查看documentation of the get
function:
def gets(chomp = true) : 字符串?
从此 IO 中读取一行。一行以 \n 字符结束。如果在此 IO 结束时调用,则返回 nil。
有多种方法可以解决它,但基本思想是您必须确保在进行 HTTP 调用时 url
不能为 nil
。例如:
url = gets
if url
# now url cannot be nil
response = HTTP::Client.get url
html = response.body
puts html
end
进一步阅读:if var
版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 dio@foxmail.com 举报,一经查实,本站将立刻删除。