如何解决JpaRepository 在保存到数据库时跳过一些行
文件长 12_500_000 行,但最后平均保存了 11_400_000 行。
我把它分成 10_000 个日志文件,并在当时 8 个线程中解析它们。
正在执行保存的服务:
@Service
public class RequestService implements InterfaceRequestService {
private final IRequestRepository repository;
public RequestService(IRequestRepository repository) {
this.repository = repository;
}
@Transactional
@Override
public void saveAll(List<Request> requests) {
for (Request r: requests) {
repository.save(r);
}
}
}
和界面:
public interface InterfaceRequestService {
void saveAll(List<Request> requests);
}
在线程中进行保存的类:
public class Importer implements Runnable{
private final IRequestService service;
int fileNumber;
public Importer(IRequestService service,int fileNumber) {
this.service = service;
this.fileNumber = fileNumber;
}
@Override
public void run() {
try {
service.saveAll(getRequestListFromFile("segment_directory/Log_segment_"+fileNumber+".txt"));
try {
Thread.sleep(10);
}catch (Exception e){
e.printstacktrace();
}
} catch (IOException e) {
e.printstacktrace();
}
}
}
public List<Request> getRequestListFromFile(String filename) throws IOException {
BufferedReader reader = new BufferedReader(new FileReader(filename));
List<Request> logs = new ArrayList<>();
String line;
while((line = reader.readLine())!=null){
logs.add(getRequestEntityFromLog(line));
}
reader.close();
return logs;
}
最后,运行所有这些的方法:
public void scheduledDataSave() throws InterruptedException {
int availableCores = Runtime.getRuntime().availableProcessors();
String directory = "segment_directory";
int filesInDirectory = Objects.requireNonNull(new File(directory).list()).length;
ExecutorService executorService = Executors.newFixedThreadPool(availableCores);
for ( int i = 1 ; i <= filesInDirectory ; i++ ) {
executorService.submit( new Importer( service,i ) );
if( i % 8 == 0 ) {
System.out.println("Sleeping");
Thread.sleep( Duration.ofSeconds( 10 ).toMillis() ) ;
}
}
executorService.shutdown();
executorService.awaitTermination(20,TimeUnit.SECONDS);
}
是否有任何理由跳过某些行?
我进行了检查
更新:
单独运行文件,但仍然没有保存所有行。
版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 dio@foxmail.com 举报,一经查实,本站将立刻删除。