Introduction

To my surprise I quite often need to read the whole contents of a file into a string. Sometimes it's easier to generate data with an external program, sometimes unittests require to read generated file, etc.

A signature of such loader function is:

std::string load_file(const std::string& path);

In C++ an offical way to deal with files are streams. There are at least two methods to load data into a string:

• use streambuf iterators to construct a string,
• use an auxilary string stream to handle a streambuf object.

std::string load1(const std::string& path) {
    std::ifstream file(path);
    return std::string((std::istreambuf_iterator<char>(file)), std::istreambuf_iterator<char>());
}


std::string load2(const std::string& path) {
    auto ss = std::ostringstream{};
    std::ifstream file(path);
    ss << file.rdbuf();
    return ss.str();
}

Both functions do their jobs, but reportedly are slow. While C++ still exposes good old C API, i.e. fread (libc) or read (POSIX), I compared performance of all solutions. Although the C solution using fread — which is shown below — is much longer than the C++ counterparts, its performance is significantly better than anything based on C++ streams. Performance of read is almost identical to fread, differences are negligible.

Of course, the performance boost highly depends on a machine type, hard drive, etc., but clearly the overhead of C++ streams is really huge compared to libc and POSIX calls.

Implementation using fread:

std::string load3(const std::string& path) {

    auto close_file = [](FILE* f){fclose(f);};

    auto holder = std::unique_ptr<FILE, decltype(close_file)>(fopen(path.c_str(), "rb"), close_file);
    if (!holder)
      return "";

    FILE* f = holder.get();

    // in C++17 following lines can be folded into std::filesystem::file_size invocation
    if (fseek(f, 0, SEEK_END) < 0)
      return "";

    const long size = ftell(f);
    if (size < 0)
      return "";

    if (fseek(f, 0, SEEK_SET) < 0)
        return "";

    std::string res;
    res.resize(size);

    // C++17 defines .data() which returns a non-const pointer
    fread(const_cast<char*>(res.data()), 1, size, f);

    return res;
}

Implementation using read:

std::string load4(const std::string& path) {

    int fd = open(path.c_str(), O_RDONLY);
    if (fd < 0)
        return "";

    struct stat sb;
    fstat(fd, &sb);

    std::string res;
    res.resize(sb.st_size);

    read(fd, const_cast<char*>(res.data()), sb.st_size);
    close(fd);

    return res;
}

Evaluation

Performance tests were run on three Linux machines. Different sizes of files were tested, each file was read 10 times and minumum times were noted.

Source code

Test programs are avilable on github.