大约 2Gb 后 C++ ofstream 写入性能下降

如何解决大约 2Gb 后 C++ ofstream 写入性能下降

我目前正在编写一个 C++ 程序，该程序需要通过按顺序写入缓冲区来在文件中写入大量数据（通常约为 5Gb）。在这个例子中，我将 400Mb 的缓冲区写入 std::ofstream。我的磁盘是 SSD，几乎是空的。 4 个缓冲区后，写入性能下降。有人知道为什么以及是否可以避免这种情况吗？

这是我的代码：

#include <iostream>
#include <iomanip>
#include <fstream>
#include <chrono>

int main()
{
  unsigned int bufferSize = 4e8; //400Mb
  unsigned char* buffer = new unsigned char[bufferSize];
  for (unsigned int i = 0; i < bufferSize; i++)
    buffer[i] = (unsigned char) i % 256; // just "randomly" writing the buffer
  std::ofstream ofs("test.bin");
  std::chrono::steady_clock::time_point start_time;
  std::chrono::steady_clock::time_point stop_time;
  std::chrono::duration<double> duration;

  for (int i = 1; i <= 10; i++)
  {
    start_time = std::chrono::steady_clock::Now();
    ofs.write((char*) buffer,bufferSize);
    stop_time = std::chrono::steady_clock::Now();
    duration = stop_time - start_time;
    std::cout << "i = " << i << ",time spent copying the buffer = " << std::chrono::duration_cast<std::chrono::nanoseconds>(duration).count() / 1e6  << "ms" << std::endl;
  }

  ofs.close();
  delete[] buffer;
  return 0;
}

这是我运行它时得到的：

nirupamix@machine:~/$ ./main.out
i = 1,time spent copying the buffer = 166.267ms
i = 2,time spent copying the buffer = 170.698ms
i = 3,time spent copying the buffer = 177.484ms
i = 4,time spent copying the buffer = 210.693ms
i = 5,time spent copying the buffer = 475.933ms
i = 6,time spent copying the buffer = 793.295ms
i = 7,time spent copying the buffer = 822.195ms
i = 8,time spent copying the buffer = 828.539ms
i = 9,time spent copying the buffer = 850.651ms
i = 10,time spent copying the buffer = 794.542ms

感谢您的时间！

解决方法

向SSD写入大量数据时可能会出现几个问题

Termal throttling - 当数据写入时间足够长时，SDD 会升温并变慢
- 您的数据不应该足够大。
SSD 缓存已满
- 根据您的 SSD 的年龄和技术，您的 SSD 可能有一些 DRAM 来缓存写入，然后再进入较慢的 SLC 缓存，最后是否已满实际 QLC 存储。（详情请见 anandtech。