SSE: reinterpret_cast<__m128*> instead of _mm_load_ps - c++

Question

I am in the process of coding up a simple convolution function in C++, starting from the very basic "sliding-window" convolution with regular products (no FFT stuff for now), up to SEE, AVX and possibly OpenCL. I ran into a problem with SSE though. My code looks like this:
for (x = 0; x < SIZEX - KSIZEX + 1; ++x)
{
for (y = 0; y < SIZEY - KSIZEY + 1; ++y)
{
tmp = 0.0f;
float fDPtmp = 0.0f;
float *Kp = &K[0];
for (xi = 0; xi < KSIZEX; ++xi, Kp=Kp+4)
{
float *Cp = &C[(x+xi)*SIZEY + y];
__m128 *KpSSE = reinterpret_cast<__m128*>(&K);
__m128 *CpSSE = reinterpret_cast<__m128*>(&C[(x + xi)*SIZEY + y]);
__m128 DPtmp = _mm_dp_ps(*KpSSE, *CpSSE, 0xFF);
_mm_store_ss(&fDPtmp, DPtmp);
tmp += fDPtmp;
}
R[k] = tmp;
++k;
}
}
The necessary matrices are initialized like this (the size of those is considerd ok because the simpler implementations work just fine):
__declspec(align(16)) float *C = ReadMatrix("E:\\Code\\conv\\C.bin");
__declspec(align(16)) float *K = ReadMatrix("E:\\Code\\conv\\K.bin");
__declspec(align(16)) float *R = new float[CSIZEX*CSIZEY];
The code crashes at y=1 so I feel there might be a mistake with the way I handle the pointers. The interesting thing is that if I replace the reinterpret_casts with _mm_set_ps, i.e.
__m128 KpSSE = _mm_set_ps(Kp[0], Kp[1], Kp[2], Kp[3]);
__m128 CpSSE = _mm_set_ps(Cp[0], Cp[1], Cp[2], Cp[3]);
__m128 DPtmp = _mm_dp_ps(KpSSE, CpSSE, 0xFF);
_mm_store_ss(&fDPtmp, DPtmp);
the whole thing works just fine although slower, which I blame on all the copy operations.
Can anybody please point me to what exactly I am doing wrong here?
Thank you very much
Pat
Update: Ok, so as pointed out by Paul the problem lies with ReadMatrix (or another solution would be to use _mm_loadu_ps). As for ReadMatrix(), it looks like this:
__declspec(align(16)) float* ReadMatrix(string path)
{
streampos size;
ifstream file(path, ios::in | ios::binary | ios::ate);
if (file.is_open())
{
size = file.tellg();
__declspec(align(16)) float *C = new float[size];
file.seekg(0, ios::beg);
file.read(reinterpret_cast<char*>(&C[0]), size);
file.close();
return C;
}
else cout << "Unable to open file" << endl;
}
It does not do the trick. Is there any other way of doing this elegantly rather than being forced to read the file piece by piece and perform memcpy, which I assume should work?!
Update:
Still does not seem to want to work after
__declspec(align(16)) float* ReadMatrix(string path)
{
streampos size;
ifstream file(path, ios::in | ios::binary | ios::ate);
if (file.is_open())
{
size = file.tellg();
__declspec(align(16)) float *C = static_cast<__declspec(align(16)) float*>(_aligned_malloc(size * sizeof(*C), 16));
file.seekg(0, ios::beg);
file.read(reinterpret_cast<char*>(&C[0]), size);
file.close();
return C;
}
else cout << "Unable to open file" << endl;
}
I added the static_cast up there since it seemed necessary to get Paul's code to compile (i.e. _aligned_malloc returns a void pointer). I am getting close to just read chunks of the file with fread and memcpy them into an alligned array. :/ Yet again I am finding myself asking for advice. Thank you very much all.
Pat
PS: Non-SSE code works fine with these data structures. _mm_loadu_ps is slower than using the non-SSE code.

Answer 1

This doesn't do what you think it does:
__declspec(align(16)) float *C = ReadMatrix("E:\\Code\\conv\\C.bin");
All that the alignment directive achieves here is to align the pointer itself (i.e. C) to a 16 byte boundary, not the contents of the pointer.
You either need to fix ReadMatrix so that it returns suitably aligned data, or use _mm_loadu_ps, as others have already suggested.
Do not use _mm_set_ps as this will tend to generate a lot of instructions under the hood, unlike _mm_loadu_ps, which maps to a single instruction.
UPDATE
You have repeated the same mistake in ReadMatrix:
__declspec(align(16)) float *C = new float[size];
again this does not guarantee the alignment of the data, only of the pointer C itself. To fix this allocation you can use _mm_malloc or _aligned_malloc:
float *C = _mm_malloc(size * sizeof(*C), 16);
or
float *C = _aligned_malloc(size * sizeof(*C), 16);

Answer 2

In ReadMatrix, you have no guarantee whatsoever that the new expression returns a properly aligned pointer. It doesn't matter that you assign to an aligned pointer (and I'm not even sure if your syntax means the pointer itself is aligned, or what it points to).
You need to use _mm_align, or _mm_malloc, or some other aligned allocation facility.

Answer 3

You can't use reinterpret_cast here, and I understand _mmloadu_ps is slow. But there is another way. Unroll your loop, read in aligned data, and shift and mask in the new value before you perform operations on it. This will be fast and correct. That is, you can do some like this in your inner loop:
__m128i x = _mm_load_ps(p);
__m128i y = _mm_load_ps(p + sizeof(float));
__m128i z;
// do your operation on x 1st time this iteration here
z = _mm_slli_si128(y, sizeof(float) * 3);
x = _mm_srli_si128(x, sizeof(float));
x = _mm_or_si128(x, z);
// do your operation on x 2nd time this iteration here
z = _mm_slli_si128(y, sizeof(float) * 2);
x = _mm_srli_si128(x, sizeof(float) * 2);
x = _mm_or_si128(x, z);
// do your operation on x 3rd time this iteration here
z = _mm_slli_si128(y, sizeof(float));
x = _mm_srli_si128(x, sizeof(float) * 3);
x = _mm_or_si128(x, z);
// do your operation on x 4th time this iteration here
x = y; // don’t need to read in x next iteration, only y
loopCounter += 4 * sizeof(float);