Can't write HDF5 file with vector bigger than 2^13 - c++

Question

I'm using C++ & HDF5 to write a file. But run into problems with it. This is the code I use:
void fileRead::writeFile(string name, const vector<double>* data) {
int dimn = data->size();
hsize_t dim[1] = {data->size()}; //-> 2^13!!!
hid_t sid = H5Pcreate(H5P_DATASET_CREATE);
hid_t didProp = H5Screate_simple(1,dim,NULL);
H5Pset_layout(sid, H5D_COMPACT);
hid_t did = H5Dcreate(fid, name.c_str(),H5T_IEEE_F64LE, didProp, H5P_DEFAULT, sid,H5P_DEFAULT);
H5Dwrite (did, H5T_NATIVE_DOUBLE, H5S_ALL, H5S_ALL, H5P_DEFAULT, &(data->at(0)));
H5Dclose(did);
H5Sclose(didProp);
H5Pclose(sid);
}
But this gives me this error message:
HDF5-DIAG: Error detected in HDF5 (1.8.10) thread 0: #000: /pub/devel/hdf5/hdf5-1.8.10-1/src/hdf5-1.8.10/src/H5D.c line 170 in H5Dcreate2(): unable to create dataset
major: Dataset
minor: Unable to initialize object #001: /pub/devel/hdf5/hdf5-1.8.10-1/src/hdf5-1.8.10/src/H5Dint.c line 439 in H5D__create_named(): unable to create and link to dataset
major: Dataset
minor: Unable to initialize object #002: /pub/devel/hdf5/hdf5-1.8.10-1/src/hdf5-1.8.10/src/H5L.c line 1638 in H5L_link_object(): unable to create new link to object
major: Links
minor: Unable to initialize object #003: /pub/devel/hdf5/hdf5-1.8.10-1/src/hdf5-1.8.10/src/H5L.c line 1882 in H5L_create_real(): can't insert link
major: Symbol table
minor: Unable to insert object #004: /pub/devel/hdf5/hdf5-1.8.10-1/src/hdf5-1.8.10/src/H5Gtraverse.c line 861 in H5G_traverse(): internal path traversal failed
major: Symbol table
minor: Object not found #005: /pub/devel/hdf5/hdf5-1.8.10-1/src/hdf5-1.8.10/src/H5Gtraverse.c line 641 in H5G_traverse_real(): traversal operator failed
major: Symbol table
minor: Callback failed #006: /pub/devel/hdf5/hdf5-1.8.10-1/src/hdf5-1.8.10/src/H5L.c line 1685 in H5L_link_cb(): unable to create object
major: Object header
minor: Unable to initialize object #007: /pub/devel/hdf5/hdf5-1.8.10-1/src/hdf5-1.8.10/src/H5O.c line 3015 in H5O_obj_create(): unable to open object
major: Object header
minor: Can't open object #008: /pub/devel/hdf5/hdf5-1.8.10-1/src/hdf5-1.8.10/src/H5Doh.c line 293 in H5O__dset_create(): unable to create dataset
major: Dataset
minor: Unable to initialize object #009: /pub/devel/hdf5/hdf5-1.8.10-1/src/hdf5-1.8.10/src/H5Dint.c line 1044 in H5D__create(): unable to construct layout information
major: Dataset
minor: Unable to initialize object #010: /pub/devel/hdf5/hdf5-1.8.10-1/src/hdf5-1.8.10/src/H5Dcompact.c line 212 in H5D__compact_construct(): compact dataset size is bigger than header message maximum size
major: Dataset
minor: Unable to initialize object HDF5-DIAG: Error detected in HDF5 (1.8.10) thread 0: #000: /pub/devel/hdf5/hdf5-1.8.10-1/src/hdf5-1.8.10/src/H5D.c line 391 in H5Dclose(): not a dataset
major: Invalid arguments to routine
minor: Inappropriate type
This happens for all vector sizes >= 2^13 (8192). Which is puzzeling me since reading in is no problem with bigger files and 2^13 is still a rather small number so something must be fishi with my code.
Any help would be appreatiated.
yours
magu_

Answer 1

From the documentation for the H5D_COMPACT parameter for H5Pset_layout:
Store raw data in the dataset object header in file. This should only
be used for datasets with small amounts of raw data. The raw data size
limit is 64K (65520 bytes). Attempting to create a dataset with raw
data larger than this limit will cause the H5Dcreate call to fail.
So if your doubles are 8 bytes, you've run into that limit.
You need to use one of the other storage options, contiguous or chunked.