当 CSS 被禁用时，HTMLUnit 不会执行 JavaScript 并加载所有内容

分类：编程问答

如何解决当 CSS 被禁用时，HTMLUnit 不会执行 JavaScript 并加载所有内容

我一直在尝试使用 HTMLUnit 从 ethermine.org 上的矿工地址仪表板中抓取页面数据。一个示例 URL 是：

https://ethermine.org/miners/32885f190a878d6db68cbC2889a3FF3abE3C257C/dashboard

调用 webClient.getOptions().setCssEnabled(false); 会阻止 javascript 加载表数据。此方法的文档是：

启用/禁用 CSS 支持。默认情况下，启用此属性。如果禁用 HtmlUnit 将不会下载链接的 css 文件，也不会触发相关的 onload/onerror 事件。

所以，我假设发生的事情是仅在 CSS 成功加载时才执行 Javascript。启用 CSS 会使我的用例加载页面时间过长。所以，我在问是否有一种方法可以让 Javascript 运行以从此页面加载表数据，同时禁用 CSS。也许这可以通过触发 onload 事件来完成？如何才能做到这一点。谢谢

这是代码和日志输出：

public class Main {


    public static void main(String[] args) {
        WebClient webClient = new WebClient();
        webClient.getOptions().setCssEnabled(false); // comment this line out and the active worker table data loads just fine
        webClient.setAjaxController(new NicelyResynchronizingAjaxController());

        try {

            WebRequest request = new WebRequest(new URL("https://ethermine.org/miners/32885f190a878d6db68cbC2889a3FF3abE3C257C/dashboard"));
            HtmlPage page = webClient.getPage(request);

            webClient.waitForBackgroundJavaScript(5000); // long enough for all javascript to execute

            System.out.println(page.asText());

        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

这是禁用 CSS 的输出：

Feb 08,2021 5:17:09 PM com.gargoylesoftware.htmlunit.WebConsole error
SEVERE: TypeError: Cannot find function get in object .
Feb 08,2021 5:17:09 PM com.gargoylesoftware.htmlunit.WebConsole error
SEVERE: TypeError: Cannot find function isEqualNode in object [object HTMLLinkElement].
Feb 08,2021 5:17:10 PM com.gargoylesoftware.htmlunit.NicelyResynchronizingAjaxController processSynchron
INFO: Re-synchronized call to https://ethermine.org/css/chunk-vendors.391ed6f9.css
Feb 08,2021 5:17:10 PM com.gargoylesoftware.htmlunit.NicelyResynchronizingAjaxController processSynchron
INFO: Re-synchronized call to https://ethermine.org/css/app.69e6b25a.css
Feb 08,2021 5:17:10 PM com.gargoylesoftware.htmlunit.NicelyResynchronizingAjaxController processSynchron
INFO: Re-synchronized call to https://ethermine.org/css/Dashboard.eb03c99b.css
Feb 08,2021 5:17:11 PM com.gargoylesoftware.htmlunit.WebConsole warn
WARNING: cssVars(): variable "--fa-primary-color" is undefined
Feb 08,2021 5:17:11 PM com.gargoylesoftware.htmlunit.WebConsole warn
WARNING: cssVars(): variable "--fa-primary-opacity" is undefined
Feb 08,2021 5:17:11 PM com.gargoylesoftware.htmlunit.WebConsole warn
WARNING: cssVars(): variable "--fa-secondary-color" is undefined
Feb 08,2021 5:17:11 PM com.gargoylesoftware.htmlunit.WebConsole warn
WARNING: cssVars(): variable "--fa-secondary-opacity" is undefined
Feb 08,2021 5:17:11 PM com.gargoylesoftware.htmlunit.WebConsole warn
WARNING: cssVars(): variable "--fa-primary-opacity" is undefined
Home - Ethermine - Ethereum (ETH) mining pool
ETH
ETH
ZEC
YEC
ETC
BEAM
RVN
unchecked
Home
Statistics
Luck
Support
Start Mining
News
News
Legal Notices
ImprintPrivacy PolicyTerms of Operation
Resources
Help Desk3rd Party AppsAPI
Pools
ethermine.orgetc.ethermine.orgzcash.flypool.orgycash.flypool.orgbeam.flypool.orgravencoin.flypool.org
Other Platforms
etherchain.orgpoolwatch.iozcha.inyec.zcha.inbeaconcha.in

这是启用 CSS 的日志输出：

Feb 08,2021 5:18:02 PM com.gargoylesoftware.htmlunit.WebConsole error
SEVERE: TypeError: Cannot find function get in object .
Feb 08,2021 5:18:02 PM com.gargoylesoftware.htmlunit.WebConsole error
SEVERE: TypeError: Cannot find function isEqualNode in object [object HTMLLinkElement].
Feb 08,2021 5:18:03 PM com.gargoylesoftware.htmlunit.NicelyResynchronizingAjaxController processSynchron
INFO: Re-synchronized call to https://api.ethermine.org/miner/32885f190a878d6db68cbC2889a3FF3abE3C257C/dashboard
Feb 08,2021 5:18:03 PM com.gargoylesoftware.htmlunit.NicelyResynchronizingAjaxController processSynchron
INFO: Re-synchronized call to https://api.ethermine.org/miner/32885f190a878d6db68cbC2889a3FF3abE3C257C/dashboard/payouts
Feb 08,2021 5:18:03 PM com.gargoylesoftware.htmlunit.NicelyResynchronizingAjaxController processSynchron
INFO: Re-synchronized call to https://api.ethermine.org/poolStats
Feb 08,2021 5:18:03 PM com.gargoylesoftware.htmlunit.NicelyResynchronizingAjaxController processSynchron
INFO: Re-synchronized call to https://api.ethermine.org/miner/32885f190a878d6db68cbC2889a3FF3abE3C257C/settings
Feb 08,2021 5:18:03 PM com.gargoylesoftware.htmlunit.NicelyResynchronizingAjaxController processSynchron
INFO: Re-synchronized call to https://api.ethermine.org/miner/32885f190a878d6db68cbC2889a3FF3abE3C257C/dashboard/msg
Feb 08,2021 5:18:07 PM com.gargoylesoftware.htmlunit.NicelyResynchronizingAjaxController processSynchron
INFO: Re-synchronized call to https://ethermine.org/css/chunk-vendors.391ed6f9.css
Feb 08,2021 5:18:07 PM com.gargoylesoftware.htmlunit.NicelyResynchronizingAjaxController processSynchron
INFO: Re-synchronized call to https://ethermine.org/css/app.69e6b25a.css
Feb 08,2021 5:18:07 PM com.gargoylesoftware.htmlunit.NicelyResynchronizingAjaxController processSynchron
INFO: Re-synchronized call to https://ethermine.org/css/Dashboard.eb03c99b.css
Feb 08,2021 5:18:08 PM com.gargoylesoftware.htmlunit.WebConsole warn
WARNING: cssVars(): variable "--fa-primary-color" is undefined
Feb 08,2021 5:18:08 PM com.gargoylesoftware.htmlunit.WebConsole warn
WARNING: cssVars(): variable "--fa-primary-opacity" is undefined
Feb 08,2021 5:18:08 PM com.gargoylesoftware.htmlunit.WebConsole warn
WARNING: cssVars(): variable "--fa-secondary-color" is undefined
Feb 08,2021 5:18:08 PM com.gargoylesoftware.htmlunit.WebConsole warn
WARNING: cssVars(): variable "--fa-secondary-opacity" is undefined
Feb 08,2021 5:18:08 PM com.gargoylesoftware.htmlunit.WebConsole warn
WARNING: cssVars(): variable "--fa-primary-opacity" is undefined
Home - Ethermine - Ethereum (ETH) mining pool
ETH
ZEC
YEC
ETC
BEAM
RVN
unchecked
Home
Statistics
Luck
Support
Start Mining
News
0x32885f190a878d6db68cbC2889a3FF3abE3C257C
Dashboard
Payouts
Settings
Help Desk
Auto-Updateunchecked
Workers Active / Inactive
0/0
Unpaid Balance
0.00012 ETH
Estimated Earnings
0.00016 ETH
Daily
Hashrate
Current (GH/s)
0.0
Average (GH/s)
0.0
Reported (GH/s)
0.0
Shares
Valid 99%
0
Stale 1%
0
Invalid
0
Created with Highcharts 7.1.2HashrateHashrateCurrent HashrateAverage HashrateReported Hashrate18:0020:0022:008. Feb02:0004:0006:0008:0010:0012:0014:0016:0010.0 GH/s10.2 GH/s10.4 GH/s10.6 GH/s10.8 GH/s
Created with Highcharts 7.1.2SharesWorkersShares & WorkersValid SharesWorkersStale Shares2021
Active Workers (57)
Name    Reported Hashrate   Current Hashrate    Valid Shares    Stale Shares    Invalid Shares  Last seen
1-1 183.7   188.8   168 3   0   8 minutes ago
1-2 183.8   188.8   168 3   0   9 minutes ago
1-3 183.8   161.8   145 1   0   8 minutes ago
1-4 183.8   179.9   160 3   0   8 minutes ago
10-1    192.2   182.2   162 3   0   9 minutes ago
10-2    194.5   192.9   173 1   0   8 minutes ago
10-3    192.3   196.3   176 1   0   8 minutes ago
10-4    189.3   209.2   185 5   0   9 minutes ago
11-1    192.0   179.2   158 5   0   9 minutes ago
11-2    168.9   149.9   133 3   0   8 minutes ago
11-3    194.8   177.7   158 3   0   8 minutes ago
11-4    198.6   163.7   146 2   0   9 minutes ago
12-1    190.9   207.8   187 0   0   9 minutes ago
12-2    163.9   169.9   151 3   0   8 minutes ago
12-3    170.1   188.5   169 1   0   9 minutes ago
12-4    170.2   145.9   130 2   0   8 minutes ago
13-1    167.9   182.2   162 3   0   8 minutes ago
13-2    182.4   157.4   141 1   0   8 minutes ago
13-3    186.4   187.0   167 2   0   8 minutes ago
13-4    166.2   156.3   140 1   0   9 minutes ago
14-1    202.2   185.8   164 5   0   8 minutes ago
14-2    269.4   283.3   253 3   0   8 minutes ago
15-1    265.2   245.7   214 11  0   8 minutes ago
15-2    167.8   186.7   168 0   0   8 minutes ago
15-3    161.3   154.4   139 0   0   8 minutes ago
2-1 181.5   164.4   146 3   0   8 minutes ago
2-2 174.7   181.4   162 2   0   8 minutes ago
2-3 188.3   188.4   167 4   0   8 minutes ago
2-4 188.3   198.1   175 5   0   8 minutes ago
3-1 192.6   170.3   152 2   0   8 minutes ago
3-2 183.7   179.6   161 1   0   9 minutes ago
3-3 188.3   189.6   170 1   0   8 minutes ago
3-4 164.4   162.5   143 5   0   9 minutes ago
4-1 168.4   150.0   135 0   0   8 minutes ago
4-2 192.7   191.1   172 0   0   8 minutes ago
4-3 192.5   211.1   190 0   0   8 minutes ago
4-4 192.4   182.9   164 1   0   8 minutes ago
5-1 168.4   176.6   157 3   0   9 minutes ago
5-2 192.5   210.3   188 2   0   9 minutes ago
5-3 171.3   183.2   161 6   0   8 minutes ago
5-4 193.5   178.5   160 1   0   8 minutes ago
6-1 140.4   145.2   130 1   0   8 minutes ago
6-2 190.9   181.1   163 0   0   8 minutes ago
6-3 191.6   206.7   186 0   0   8 minutes ago
6-4 190.4   185.1   164 4   0   8 minutes ago
7-1 194.5   186.3   167 1   0   8 minutes ago
7-2 193.2   179.2   160 2   0   8 minutes ago
7-3 197.2   208.1   186 2   0   8 minutes ago
7-4 194.7   187.0   167 2   0   8 minutes ago
8-1 193.1   188.5   169 1   0   8 minutes ago
8-2 192.3   187.0   167 2   0   8 minutes ago
8-3 146.0   145.4   127 6   0   8 minutes ago
8-4 196.4   218.1   195 2   0   8 minutes ago
9-1 194.6   207.8   187 0   0   8 minutes ago
9-2 191.7   167.8   151 0   0   10 minutes ago
9-3 152.4   151.4   135 2   0   9 minutes ago
9-4 194.2   207.7   185 3   0   9 minutes ago
Legal Notices
ImprintPrivacy PolicyTerms of Operation
Resources
Help Desk3rd Party AppsAPI
Pools
ethermine.orgetc.ethermine.orgzcash.flypool.orgycash.flypool.orgbeam.flypool.orgravencoin.flypool.org
Other Platforms
etherchain.orgpoolwatch.iozcha.inyec.zcha.inbeaconcha.in

解决方法

所以，我假设发生的事情是仅在 CSS 成功加载时才执行 Javascript。

那是错误的 - 没有这样的依赖。

也许 js 代码停止了，因为它依赖于一些 css 东西的存在。检查您的日志，请尝试添加

webClient.getOptions().setThrowExceptionOnScriptError(false);

在第一个错误时不停止 js 执行。 如果这没有帮助，请在 GitHub 上打开一个问题。

版权声明：本文内容由互联网用户自发贡献，该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容， 请发送邮件至 dio@foxmail.com 举报，一经查实，本站将立刻删除。

相关推荐

matplotlib报错：AttributeError: module 'backend_interagg' has no attribute 'FigureCanvas'. Did you mean: 'FigureCanvasAgg'?
使用本地python环境可以成功执行 import pandas as pd import matplotlib.pyplot as plt # 设置字体 plt.rcParams['font.sans-serif'] = ['SimHei'] # 能正确显示负号 p
gitlab登录失败，报错：This challenge page was accidentally cached by an intermediary and is no longer available.
设置时间 控制面板
后端开发常见错误
错误1：Request method ‘DELETE‘ not supported 错误还原：controller层有一个接口，访问该接口时报错：Request method ‘DELETE‘ not supported 错误原因：没有接收到前端传入的参数，修改为如下 参考 错误2：cannot r
docker常见错误
错误1：启动docker镜像时报错：Error response from daemon: driver failed programming external connectivity on endpoint quirky_allen 解决方法：重启docker -> systemctl r
idea常见错误
错误1：private field ‘xxx‘ is never assigned 按Altʾnter快捷键，选择第2项 参考：https://blog.csdn.net/shi_hong_fei_hei/article/details/88814070 错误2：启动时报错，不能找到主启动类 #
pip安装依赖失败
报错如下，通过源不能下载，最后警告pip需升级版本 Requirement already satisfied: pip in c:\users\ychen\appdata\local\programs\python\python310\lib\site-packages (22.0.4) Coll
maven常见错误
错误1：maven打包报错 错误还原：使用maven打包项目时报错如下 [ERROR] Failed to execute goal org.apache.maven.plugins:maven-resources-plugin:3.2.0:resources (default-resources)
cloud项目中常见错误
错误1：服务调用时报错 服务消费者模块assess通过openFeign调用服务提供者模块hires 如下为服务提供者模块hires的控制层接口 @RestController @RequestMapping("/hires") public class FeignControl
springboot常见错误
错误1：运行项目后报如下错误 解决方案 报错2：Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.1:compile (default-compile) on project sb 解决方案：在pom.
springmvc拦截器中使用redisTemplate报空指针异常
参考 错误原因 过滤器或拦截器在生效时，redisTemplate还没有注入 解决方案：在注入容器时就生效 @Component //项目运行时就注入Spring容器 public class RedisBean { @Resource private RedisTemplate<String
vite报错：npm ERR! command C:WINDOWSsystem32cmd.exe /d /s /c C:UsersychenAppDataLocalTempnpx-dde1c848.cmd
使用vite构建项目报错 C:\Users\ychen\work>npm init @vitejs/app @vitejs/create-app is deprecated, use npm init vite instead C:\Users\ychen\AppData\Local\npm-
VM VirtualBox中安装centos时报错：设置基础软件仓库时出错
参考1 参考2 解决方案 # 点击安装源 协议选择 http:// 路径填写 mirrors.aliyun.com/centos/8.3.2011/BaseOS/x86_64/os URL类型 软件库URL 其他路径 # 版本 7 mirrors.aliyun.com/centos/7/os/x86
kafka常见问题
报错1 [root@slave1 data_mocker]# kafka-console-consumer.sh --bootstrap-server slave1:9092 --topic topic_db [2023-12-19 18:31:12,770] WARN [Consumer clie
hive常见问题
错误1 # 重写数据 hive (edu)> insert overwrite table dwd_trade_cart_add_inc > select data.id, > data.user_id, > data.course_id, > date_format(
装载数据时报错：Failed to execute spark task, with exception 'org.apache.hadoop.hive.ql.metadata.HiveException(Failed to create spark client.)'
错误1 hive (edu)> insert into huanhuan values(1,'haoge'); Query ID = root_20240110071417_fe1517ad-3607-41f4-bdcf-d00b98ac443e Total jobs = 1
flume常见问题
报错1：执行到如下就不执行了，没有显示Successfully registered new MBean. [root@slave1 bin]# /usr/local/software/flume-1.9.0/bin/flume-ng agent -n a1 -c /usr/local/softwa
hadoop集群常见错误
虚拟及没有启动任何服务器查看jps会显示jps，如果没有显示任何东西 [root@slave2 ~]# jps 9647 Jps 解决方案 # 进入/tmp查看 [root@slave1 dfs]# cd /tmp [root@slave1 tmp]# ll 总用量 48 drwxr-xr-x. 2
hive常见错误
报错1 hive> show databases; OK Failed with exception java.io.IOException:java.lang.RuntimeException: Error in configuring object Time taken: 0.474 se
Linux常见错误
报错1 [root@localhost ~]# vim -bash: vim: 未找到命令 安装vim yum -y install vim* # 查看是否安装成功 [root@hadoop01 hadoop]# rpm -qa |grep vim vim-X11-7.4.629-8.el7_9.x
使用sqoop一直卡在：mapreduce.Job: Running job: job_1703173956074_0001
修改hadoop配置 vi /usr/local/software/hadoop-2.9.2/etc/hadoop/yarn-site.xml # 添加如下 <configuration> <property> <name>yarn.nodemanager.res
pythonJavaScriptjavaHTMLPHPreactjsC#AndroidCSSNode.jssqlrpython-3.xMysqLjQueryc++pandasFlutterangularIOSdjangolinuxswifttypescript路由器JSON路由器设置无线路由器h3c华三华三路由器设置华三路由器电脑软件教程arraysdocker软件图文教程Cvue.jslaravelspring-boot