如何解决使用--profile slurm
我正在使用snakemake创建一个管道来调用纳米孔测序数据中的甲基化。我已经使用--dryrun选项运行了snakenake,并且dag已成功构建。但是,当我添加选项--profile slurm时,出现以下错误:
(nanopolish) [danielle.perley@talonhead2 nanopolish-CpG-calling]$ snakemake -np --use-conda --profile slurm test_data/20-001-002/20-001-002_fastq_pass.gz
Building DAG of jobs...
Job counts:
count jobs
1 combine_tech_reps
1
InputFunctionException in line 32 of /home/danielle.perley/nanopolish-CpG-calling/Snakefile:
Error:
SyntaxError: invalid syntax (<string>,line 1)
Wildcards:
sample=20-001-002
Traceback:
File "/home/danielle.perley/miniconda3/envs/nanopolish/lib/python3.7/site-packages/snakemake/executors/__init__.py",line 115,in run_jobs
File "/home/danielle.perley/miniconda3/envs/nanopolish/lib/python3.7/site-packages/snakemake/executors/__init__.py",line 120,in run
File "/home/danielle.perley/miniconda3/envs/nanopolish/lib/python3.7/site-packages/snakemake/executors/__init__.py",line 131,in _run
File "/home/danielle.perley/miniconda3/envs/nanopolish/lib/python3.7/site-packages/snakemake/executors/__init__.py",line 151,in printjob
File "/home/danielle.perley/miniconda3/envs/nanopolish/lib/python3.7/site-packages/snakemake/executors/__init__.py",line 137,in printjob
第33行是我的snakefile中的rule Combine_tech_reps。 (我只在这里显示我的蛇文件的第一部分)
from snakemake.utils import validate
import pandas as pd
import os.path
import glob
configfile: "config.yaml"
samples_df = pd.read_table(config["samples"],sep = '\t')
samples_df = samples_df.set_index("Sample")
samples = list(samples_df.index.unique())
wildcard_constraints:
sample = "|".join(samples)
def get_fast5(wildcards):
f5 = glob.glob(os.path.join(config["raw_data"],wildcards.sample,"2*","fast5_pass"))
return(f5)
localrules: all,build_index
rule all:
input:
expand("results/Methylation/{sample}_frequency.tsv",sample=samples),expand("results/alignments/{sample}_flagstat.txt",expand("resources/QC/{sample}_pycoQC.json",expand("results/QC/{sample}_pycoQC.html","report/multiQC.html"
rule combine_tech_reps:
input:
fqs = lambda wildcards: glob.glob(os.path.join(config["raw_data"],"{sample}","{sample}_fastq_pass.gz").format(sample=wildcards.sample))
output:
fq = os.path.join(config["raw_data"],"{sample}_fastq_pass.gz")
shell: """
zcat {input} > {output}
"""
我在目录中有一个配置文件: 〜/ .config / snakemake / slurm / config.yaml
jobs: 10
cluster: "sbatch -p talon -t {resources.time} --mem={resources.mem} -c {resources.cpus} -o logs_slurm/{rule}_{wildcards} -e logs_slurm/{rule}_{wildcards}"
default-resources: [cpus=1,mem=2000,time=10:00]
use-conda: true
我真的很想在我们的HPC上使用此管道,但是我不确定是什么导致了此错误。
解决方法
借助这篇文章,我能够解决我的问题:
InputFunctionException: unexpected EOF while parsing
通过添加详细标志:
snakemake -np --verbose --use-conda --profile slurm test_data/20-001-002/20-001-002_fastq_pass.gz
我可以看到snakemake的默认资源有问题:
10:00
^
更改config.yaml文件的默认资源行:
default-resources: [cpus=1,mem=2000,time=600]
消除了错误。
,我不确定default-resources是否是配置中的有效密钥。
如果以config.yaml的身份尝试会发生什么:
jobs: 10
cluster: "sbatch -p talon -t {resources.time} --mem={resources.mem} -c {resources.cpus} -o logs_slurm/{rule}_{wildcards} -e logs_slurm/{rule}_{wildcards}"
use-conda: true
__default__:
time: 10
cpus: 1
mem: 2GB
版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 dio@foxmail.com 举报,一经查实,本站将立刻删除。