Does anyone have a FileSystemWatcher-like class in C++/WinAPI? - c++

I need a .Net's FileSystemWatcher analog in raw C++/WinAPI.
I almost started to code one myself using FindFirstChangeNotification/FindNextChangeNotification, but then it occurred to me that I am probably not the first one who needs this and maybe someone will be willing to share.
Ideally what I need is a class which can be used as follows:
FileWatcher fw;
fw.startWatching("C:\MYDIR", "filename.dat",
FileWatcher::SIZE | FileWatcher::LAST_WRITE,
&myChangeHandler);
...
fw.stopWatching();
Or if it would use somehting like boost::signal it would be even better.
But please, no dependencies other than the Standard Library, boost and raw WinAPI.
Thanks!

What about the ReadDirectoryChangesW function?
http://msdn.microsoft.com/en-us/library/aa365465(VS.85).aspx
It stores notifications in a buffer so you don't miss any changes (unless the buffer overflows)

2021 answer:
A forked version of the repo listed below that is actively maintained: https://github.com/SpartanJ/efsw
Old answer:
This is a cross-platform solution, but does the job wrapping the Win32 stuff nicely:
https://github.com/jameswynn/simplefilewatcher

There is some public-domain code here. My current project uses this (inherited from previous developers). It works pretty well but we do miss notifications for reasons that are unclear (and possibly not caused by this code).
Note that the Win32 API here has some limitations which make it difficult/impossible to avoid missing notifications. Background and alleged work-round for the API are here

http://msdn.microsoft.com/en-us/library/system.io.filesystemwatcher.created%28v=vs.71%29.aspx the above does throgh C#, we can always write a COM Wrapper

This is an example of ReadDirectoryChangesW, written in go
kernel32dll := w32.NewKernel32DLL()
dirPath := "C://test_dir"
// Get the HANDLE of the target directory
hDir, _ := kernel32dll.CreateFile(dirPath,
w32.FILE_LIST_DIRECTORY,
w32.FILE_SHARE_READ|w32.FILE_SHARE_WRITE|w32.FILE_SHARE_DELETE,
0,
w32.OPEN_EXISTING,
w32.FILE_FLAG_BACKUP_SEMANTICS|w32.FILE_FLAG_OVERLAPPED,
0,
)
defer kernel32dll.CloseHandle(hDir) // close the handle when the program exit
var maxBufferSize uint32 = 96 // depend on you.
buffer := make([]uint8, maxBufferSize)
// a function for reset the data.
memset := func(a []uint8, v uint8) {
for i := range a {
a[i] = v
}
}
// a function for get the filename
getName := func(offset, fileNameLength uint32) string {
size := fileNameLength / 2
filename := make([]uint16, size)
var i uint32 = 0
for i = 0; i < size; i++ {
filename[i] = binary.LittleEndian.Uint16([]byte{buffer[offset+2*i], buffer[offset+2*i+1]})
}
return syscall.UTF16ToString(filename)
}
var record w32.FILE_NOTIFY_INFORMATION
for {
var dwBytes uint32 = 0
memset(buffer, 0) // clear the buffer for use again.
kernel32dll.ReadDirectoryChanges(hDir,
uintptr(unsafe.Pointer(&buffer[0])),
maxBufferSize,
true, // bWatchSubtree
w32.FILE_NOTIFY_CHANGE_LAST_WRITE|w32.FILE_NOTIFY_CHANGE_CREATION|w32.FILE_NOTIFY_CHANGE_FILE_NAME,
&dwBytes,
nil,
0,
)
if dwBytes == 0 { // if successful dwBytes is the number bytes used, or zero for Failed.
fmt.Printf("Buffer overflow! max-size:%d\n", maxBufferSize)
return
}
record = *(*w32.FILE_NOTIFY_INFORMATION)(unsafe.Pointer(&buffer[0]))
// There may be many FILE_NOTIFY_INFORMATION. For example, if you rename the file, it will trigger the FILE_ACTION_RENAMED_OLD_NAME and FILE_ACTION_RENAMED_NEW_NAM
var offsetFilename uint32 = 12 // The 12 is calculated from FILE_NOTIFY_INFORMATION.{NextEntryOffset, Action, FileName Length} => they are uint32 => 4*3=12
for {
switch record.Action {
case w32.FILE_ACTION_ADDED:
fmt.Println("FILE_ACTION_ADDED")
case w32.FILE_ACTION_REMOVED:
fmt.Println("FILE_ACTION_REMOVED")
return
case w32.FILE_ACTION_MODIFIED:
fmt.Println("FILE_ACTION_MODIFIED")
case w32.FILE_ACTION_RENAMED_OLD_NAME:
fmt.Println("FILE_ACTION_RENAMED_OLD_NAME")
case w32.FILE_ACTION_RENAMED_NEW_NAME:
fmt.Println("FILE_ACTION_RENAMED_NEW_NAME")
default:
break
}
fmt.Println(getName(offsetFilename, record.FileNameLength))
if record.NextEntryOffset == 0 {
break
}
offsetFilename = record.NextEntryOffset + 12
record = *(*w32.FILE_NOTIFY_INFORMATION)(unsafe.Pointer(uintptr(unsafe.Pointer(&buffer[0])) + uintptr(record.NextEntryOffset)))
}
}
You can go here to get the complete code.
https://github.com/CarsonSlovoka/go-pkg/blob/cf4a28372b05458d715ab118d4ce888b2727ac4d/v2/w32/kernel32_func_test.go#L465-L597

Related

Prettier putting if statement on one line

Prettier formats if statement without curley braces into one line.
This means that this :
function getErrorMessage(response) {
let errorMessage = null;
if (!response.originalError.response)
errorMessage = 'network error';
else
errorMessage = response.originalError.response.data.errorMessage;
return errorMessage;
}
becomes this :
function getErrorMessage(response) {
let errorMessage = null;
if (!response.originalError.response) errorMessage = 'network error';
else errorMessage = response.originalError.response.data.errorMessage;
return errorMessage;
}
which is FAR more unreadable.
Is there a way of disabling this?
As asked in a similar question, it turns out that the answer is that you can not and will not be able to.
As for the WFT that an average senses, well... Apparently, opinionated doesn't mean respected and well-considered in opinion of many. It means that it's implementing the author's opinion.
So, surprisingly, the unexpected thing isn't the lack of configurability but rather that there are any options to be set at all! Go figure... Someone should create a new package called EvenPrettier or FexiblyPrettier and fork in more options. If I only knew how, I'd do it.
I finally ended up using Beautify - HookyQR extension for vscode
https://marketplace.visualstudio.com/items?itemName=HookyQR.beautify
Example Settings
File: .jsbeautifyrc
{
"brace_style": "collapse,preserve-inline",
"max_preserve_newlines": 2,
"end_with_newline": false
}
Example Code
File: a.js
function dfs(start) {
if (start > n)
return;
ans.push(start);
for (let i = 0; i <= 9; i++)
dfs(start * 10 + i);
}

Where does MFC COleControl::DoPropExchange store persistent properties?

I have taken over maintenance of a legacy MFC OCX control in C++. The project is now in VS2013. I'm trying to understand the functioning of the DoPropExchange method. This method appears to be calling PX_?????(member) for nearly all the data members in the control where ???? is the type (Bool, Short, Long ...) My understanding is these are called for the purpose of providing persistent storage of properties. But from my understanding of the operation of the OCX control there are no persistent properties. Would there be any other reason to be calling PX_???? for all data members in DoPropExchange other than to support persistent properties? I'm also trying to understand where these persistent properties are loaded/stored. Where is the serialized file for loading/storing persistent property values specified?
Here is the source for DoPropExchange
// CSigPlusCtrl::DoPropExchange - Persistence support
void CSigPlusCtrl::DoPropExchange(CPropExchange* pPX)
{
DWORD Version;
long BaudRate;
short ComPort;
BOOL Rv;
LOG(("DoPropExchange Entry"));
ExchangeVersion(pPX, MAKELONG(_wVerMinor, _wVerMajor));
COleControl::DoPropExchange(pPX);
Version = pPX->GetVersion();
if (pPX->IsLoading())
{
LoadDefaultProperties();
LoadIniParameters();
}
if ((Version & 0xFFFF0000) == (DWORD)MAKELONG(0, _wVerMajor))
{
Rv = PX_Short(pPX, _T("ImageFileFormat"), ImageFileFormat, 0);
Rv = PX_Short(pPX, _T("ImageXSize"), ImageXSize, 0);
Rv = PX_Short(pPX, _T("ImageYSize"), ImageYSize, 0);
Rv = PX_Short(pPX, _T("ImagePenWidth"), ImagePenWidth, 1);
. . .
Rv = PX_Short(pPX, _T("ZoomY"), ZoomY, 0);
Rv = PX_Short(pPX, _T("ZoomPower"), ZoomPower, 1);
if (pPX->IsLoading())
{
if (SigBlob != NULL)
{
GlobalFree(SigBlob);
SigBlob = NULL;
}
}
else
{
if (SigBlob == NULL)
{
SigBlobType* SigBlobPtr;
SigBlob = GlobalAlloc(GMEM_MOVEABLE, sizeof(DWORD));
SigBlobPtr = (SigBlobType*)GlobalLock(SigBlob);
SigBlobPtr->Size = 0;
GlobalUnlock(SigBlob);
}
}
if ((Version & 0xFFFF) == Version223)
{
Rv = PX_Blob(pPX, _T("SigBlob"), SigBlob, NULL);
}
if ((Version & 0xFFFF) >= Version224)
{
CString SigStr;
if (!pPX->IsLoading())
{
SigStr = BlobToString();
}
Rv = PX_String(pPX, _T("SigStringStored"), SigStr, _T(""));
if (pPX->IsLoading())
{
BlobFromString(SigStr);
}
}
}
else
{
SigMessageBox("Warning Incompatable Versions of SigPlus Control");
}
LoadTabletParameters();
LOG(("DoPropExchange Exit"));
}
EDIT Added 6-21-2018
Running in the debugger I observe that when DoPropExchange is called, VS2013 shows the stack with a message that stack frames below may be incorrect. And the one frame just above, that calls DoPropExchange, is from mfc120d.dll which does not have symbol file available mfc120d.i386.pdb.
This Microsoft Forum Post seems to indicate that the symbol file is not available for VS2015 and I'm wondering if that is also the case for VS2013. So far I have not been able to find place to download MFC120 symbols for debug.
Starting a bounty today to find someone who can explain how and where properties are normally serialized for OLE controls and what methods are used to specify the serialized data storage location/media. This is of concern because this control runs in a Citrix ZenDesk network environment in a Terminal Aware program and if properties are being stored somewhere then each client needs to specify a location unique to that client.
The DoPropExchange is used to implement control attribute persistence mainly between design and run-time. The actual destination sink is passed by the client of the OCX.
In VC the the settings are stored in RC file while in VB in frm and frx files. If you open FRM in notepad you would probably see the section with the properties of this control.
As a side note, there is a similar implementation in case the control is used in HTML with inline settings in he html itself.
Unless your VB clients saves the settings externally via direct calls to Property bag function it is not likely you have a issue here since the above properties are not stored at runtime.

Some Problems of Indy 10 IdHTTP Implementation

In regard to Indy 10 of IdHTTP, many things have been running perfectly, but there are a few things that don't work so well here. That is why, once again, I need your help.
Download button has been running perfectly. I'm using the following code :
void __fastcall TForm1::DownloadClick(TObject *Sender)
{
MyFile = SaveDialog->FileName;
TFileStream* Fist = new TFileStream(MyFile, fmCreate | fmShareDenyNone);
Download->Enabled = false;
Urlz = Edit1->Text;
Url->Caption = Urlz;
try
{
IdHTTP->Get(Edit1->Text, Fist);
IdHTTP->Connected();
IdHTTP->Response->ResponseCode = 200;
IdHTTP->ReadTimeout = 70000;
IdHTTP->ConnectTimeout = 70000;
IdHTTP->ReuseSocket;
Fist->Position = 0;
}
__finally
{
delete Fist;
Form1->Updated();
}
}
However, a "Cancel Resume" button is still can't resume interrupted downloads. Meant, it is always sending back the entire file every time I call Get() though I've used IdHTTP->Request->Ranges property.
I use the following code:
void __fastcall TForm1::CancelResumeClick(TObject *Sender)
{
MyFile = SaveDialog->FileName;;
TFileStream* TFist = new TFileStream(MyFile, fmCreate | fmShareDenyNone);
if (IdHTTP->Connected() == true)
{
IdHTTP->Disconnect();
CancelResume->Caption = "RESUME";
IdHTTP->Response->AcceptRanges = "Bytes";
}
else
{
try {
CancelResume->Caption = "CANCEL";
// IdHTTP->Request->Ranges == "0-100";
// IdHTTP->Request->Range = Format("bytes=%d-",ARRAYOFCONST((TFist->Position)));
IdHTTP->Request->Ranges->Add()->StartPos = TFist->Position;
IdHTTP->Get(Edit1->Text, TFist);
IdHTTP->Request->Referer = Edit1->Text;
IdHTTP->ConnectTimeout = 70000;
IdHTTP->ReadTimeout = 70000;
}
__finally {
delete TFist;
}
}
Meanwhile, by using the FormatBytes function, found here, has been able to shows only the size of download files. But still unable to determine the speed of download or transfer speed.
I'm using the following code:
void __fastcall TForm1::IdHTTPWork(TObject *ASender, TWorkMode AWorkMode, __int64 AWorkCount)
{
__int64 Romeo = 0;
Romeo = IdHTTP->Response->ContentStream->Position;
// Romeo = AWorkCount;
Download->Caption = FormatBytes(Romeo) + " (" + IntToStr(Romeo) + " Bytes)";
ForSpeed->Caption = FormatBytes(Romeo);
ProgressBar->Position = AWorkCount;
ProgressBar->Update();
Form1->Updated();
}
Please advise and give an example. Any help would sure be appreciated!
In your DownloadClick() method:
Calling Connected() is useless, since you don't do anything with the result. Nor is there any guarantee that the connection will remain connected, as the server could send a Connection: close response header. I don't see anything in your code that is asking for HTTP keep-alives. Let TIdHTTP manage the connection for you.
You are forcing the Response->ResponseCode to 200. Don't do that. Respect the response code that the server actually sent. The fact that no exception was raised means the response was successful whether it is 200 or 206.
You are reading the ReuseSocket property value and ignoring it.
There is no need to reset the Fist->Position property to 0 before closing the file.
Now, with that said, your CancelResumeClick() method has many issues.
You are using the fmCreate flag when opening the file. If the file already exists, you will overwrite it from scratch, thus TFist->Position will ALWAYS be 0. Use fmOpenReadWrite instead so an existing file will open as-is. And then you have to seek to the end of the file to provide the correct Position to the Ranges header.
You are relying on the socket's Connected() state to make decisions. DO NOT do that. The connection may be gone after the previous response, or may have timed out and been closed before the new request is made. The file can still be resumed either way. HTTP is stateless. It does not matter if the socket remains open between requests, or is closed in between. Every request is self-contained. Use information provided in the previous response to govern the next request. Not the socket state.
You are modifying the value of the Response->AcceptRanges property, instead of using the value provided by the previous response. The server tells you if the file supports resuming, so you have to remember that value, or query it before then attempting to resumed download.
When you actually call Get(), the server may or may not respect the requested Range, depending on whether the requested file supports byte ranges or not. If the server responds with a response code of 206, the requested range is accepted, and the server sends ONLY the requested bytes, so you need to APPEND them to your existing file. However, if the server response with a response code of 200, the server is sending the entire file from scratch, so you need to REPLACE your existing file with the new bytes. You are not taking that into account.
In your IdHTTPWork() method, in order to calculate the download/transfer speed, you have to keep track of how many bytes are actually being transferred in between each event firing. When the event is fired, save the current AWorkCount and tick count, and then the next time the event is fired, you can compare the new AWorkCount and current ticks to know how much time has elapsed and how many bytes were transferred. From those value, you can calculate the speed, and even the estimated time remaining.
As for your progress bar, you can't use AWorkCount alone to calculate a new position. That only works if you set the progress bar's Max to AWorkCountMax in the OnWorkBegin event, and that value is not always know before a download begins. You need to take into account the size of the file being downloaded, whether it is being downloaded fresh or being resumed, how many bytes are being requested during a resume, etc. So there is lot more work involved in displaying a progress bar for a HTTP download.
Now, to answer your two questions:
How to retrieve and save the download file to a disk by using its original name?
It is provided by the server in the filename parameter of the Content-Disposition header, and/or in the name parameter of the Content-Type header. If neither value is provided by the server, you can use the filename that is in the URL you are requesting. TIdHTTP has a URL property that provides the parsed version of the last requested URL.
However, since you are creating the file locally before sending your download request, you will have to create a local file using a temp filename, and then rename the local file after the download is complete. Otherwise, use TIdHTTP.Head() to determine the real filename (you can also use it to determine if resuming is supported) before creating the local file with that filename, then use TIdHTTP.Get() to download to that local file. Otherwise, download the file to memory using TMemoryStream instead of TFileStream, and then save with the desired filename when complete.
when I click http://get.videolan.org/vlc/2.2.1/win32/vlc-2.2.1-win32.exe then the server will process requests to its actual url. http://mirror.vodien.com/videolan/vlc/2.2.1/win32/vlc-2.2.1-win32.exe. The problem is that IdHTTP will not automatically grab through it.
That is because VideoLan is not using an HTTP redirect to send clients to the real URL (TIdHTTP supports HTTP redirects). VideoLan is using an HTML redirect instead (TIdHTTP does not support HTML redirects). When a webbrowser downloads the first URL, a 5 second countdown timer is displayed before the real download then begins. As such, you will have to manually detect that the server is sending you an HTML page instead of the real file (look at the TIdHTTP.Response.ContentType property for that), parse the HTML to determine the real URL, and then download it. This also means that you cannot download the first URL directly into your target local file, otherwise you will corrupt it, especially during a resume. You have to cache the server's response first, either to a temp file or to memory, so you can analyze it before deciding how to act on it. It also means you have to remember the real URL for resuming, you cannot resume the download using the original countdown URL.
Try something more like the following instead. It does not take into account for everything mentioned above (particularly speed/progress tracking, HTML redirects, etc), but should get you a little closer:
void __fastcall TForm1::DownloadClick(TObject *Sender)
{
Urlz = Edit1->Text;
Url->Caption = Urlz;
IdHTTP->Head(Urlz);
String FileName = IdHTTP->Response->RawHeaders->Params["Content-Disposition"]["filename"];
if (FileName.IsEmpty())
{
FileName = IdHTTP->Response->RawHeaders->Params["Content-Type"]["name"];
if (FileName.IsEmpty())
FileName = IdHTTP->URL->Document;
}
SaveDialog->FileName = FileName;
if (!SaveDialog->Execute()) return;
MyFile = SaveDialog->FileName;
TFileStream* Fist = new TFileStream(MyFile, fmCreate | fmShareDenyWrite);
try
{
try
{
Download->Enabled = false;
Resume->Enabled = false;
IdHTTP->Request->Clear();
//...
IdHTTP->ReadTimeout = 70000;
IdHTTP->ConnectTimeout = 70000;
IdHTTP->Get(Urlz, Fist);
}
__finally
{
delete Fist;
Download->Enabled = true;
Updated();
}
}
catch (const EIdHTTPProtocolException &)
{
DeleteFile(MyFile);
throw;
}
}
void __fastcall TForm1::ResumeClick(TObject *Sender)
{
TFileStream* Fist = new TFileStream(MyFile, fmOpenReadWrite | fmShareDenyWrite);
try
{
Download->Enabled = false;
Resume->Enabled = false;
IdHTTP->Request->Clear();
//...
Fist->Seek(0, soEnd);
IdHTTP->Request->Ranges->Add()->StartPos = Fist->Position;
IdHTTP->Request->Referer = Edit1->Text;
IdHTTP->ConnectTimeout = 70000;
IdHTTP->ReadTimeout = 70000;
IdHTTP->Get(Urlz, Fist);
}
__finally
{
delete Fist;
Download->Enabled = true;
Updated();
}
}
void __fastcall TForm1::IdHTTPHeadersAvailable(TObject*Sender, TIdHeaderList *AHeaders, bool &VContinue)
{
Resume->Enabled = ( ((IdHTTP->Response->ResponseCode == 200) || (IdHTTP->Response->ResponseCode == 206)) && TextIsSame(AHeaders->Values["Accept-Ranges"], "bytes") );
if ((IdHTTP->Response->ContentStream) && (IdHTTP->Request->Ranges->Count > 0) && (IdHTTP->Response->ResponseCode == 200))
IdHTTP->Response->ContentStream->Size = 0;
}
#Romeo:
Also, you can try a following function to determine the real download filename.
I've translated this to C++ based on the RRUZ'function. So far so good, I'm using it on my simple IdHTTP download program, too.
But, this translation result is of course still need value improvement input from Remy Lebeau, RRUZ, or any other master here.
String __fastcall GetRemoteFileName(const String URI)
{
String result;
try
{
TIdHTTP* HTTP = new TIdHTTP(NULL);
try
{
HTTP->Head(URI);
result = HTTP->Response->RawHeaders->Params["Content-Disposition"]["filename"];
if (result.IsEmpty())
{
result = HTTP->Response->RawHeaders->Params["Content-Type"]["name"];
if (result.IsEmpty())
result = HTTP->URL->Document;
}
}
__finally
{
delete HTTP;
}
}
catch(const Exception &ex)
{
ShowMessage(const_cast<Exception&>(ex).ToString());
}
return result;
}

Show/Add an Item from/to Some ComboBoxs in C++ Builder XE8

I need your help to resolve my 2 problems.
I'm using 3 ComboBoxs (CB). When choosing an item from 1st CB, then an item of 2nd & 3rd CB will automatically come up .
On Delphi, all settings are work so well via "Text" property.
1st CB contains:
Google
Yahoo
2nd CB contains:
8.8.8.8
9.9.9.9
3rd CB contains:
8888
9999
Below is my code on Delphi :
procedure TForm.1stCBChange(Sender: TObject);
begin
if 1stCB.Text = 'Google' then begin
2ndCB.Text := '8.8.8.8';
3rdCB.Text := '8888';
end else
if 1stCB.Text = 'Yahoo' then begin
2ndCB.Text := '9.9.9.9';
3rdCB.Text := '9999';
end;
end;
The problem is that on C++ Builder XE8, it's not work properly via "Text" property. On the other words, the 2nd & 3rd CB's item is not shown via "Text" property.
void __fastcall TForm::1stCBChange(TObject *Sender)
{
if (1stCBChange->Text == "Google")
{
2ndCB->Text = "8.8.8.8";
3rdCB->Text = "8888";
}
else
if (1stCBChange->Text == "Yahoo")
{
2ndCB->Text = "9.9.9.9";
3rdCB->Text = "9999";
}
}
So to make it works, I have to use "ItemIndex" property.
void __fastcall TForm::1stCBChange(TObject *Sender)
{
if (1stCBChange->ItemIndex == 0)
{
2ndCB->ItemIndex = IntToStr(0);
// or 2ndCB->ItemIndex = 0;
3rdCB->ItemIndex = IntToStr(0);
}
else
if (1stCBChange->ItemIndex == 1)
{
2ndCB->ItemIndex = IntToStr(1);
3rdCB->ItemIndex = IntToStr(1);
}
}
I've tried some ways, but not works, e.g. 2ndCB->Items->Objects[1] = (TObject*) new String("8.8.8.8"); or 2ndCB->Items->AddObject("8.8.8.8",2ndCB); so on and on...
So my question: how to make it works on C++ Builder XE8 via "Text" option really like on Delphi ?
Thank you very much for your help.
About your first problem: you probably don't have the same Style property on the Combobox in Delphi and C++Builder. Setting it to csDropDown allows you to specify the Text, but csDropDownList doesn't.
Now, ItemIndex is an integer property, so the call to IntToStr is not needed. It shouldn't even compile, as you can't assign text to an integer variable, and in XE5 at least it doesn't. If it works later it must be because there is an operator int() or similar for UnicodeString, although I didn't find it on the latest documentation.
About your second issue: Add() adds the passed string to the end of the list. If you want to insert the string in a specific position, then you need to call Insert() which asks you for the position to insert it, in your case it would be 0. This, of course, assumes the StringList is not sorted.
Here is the Documentation for TStrings: http://docwiki.embarcadero.com/Libraries/XE5/en/System.Classes.TStrings

AppFabric Cache concurrency issue?

While stress testing prototype of our brand new primary system, I run into concurrent issue with AppFabric Cache. When concurrently calling many DataCache.Get() and Put() with same cacheKey, where I attempt to store relatively large objet, I recieve "ErrorCode:SubStatus:There is a temporary failure. Please retry later." It is reproducible by the following code:
var dcfc = new DataCacheFactoryConfiguration
{
Servers = new[] {new DataCacheServerEndpoint("localhost", 22233)},
SecurityProperties = new DataCacheSecurity(DataCacheSecurityMode.None, DataCacheProtectionLevel.None),
};
var dcf = new DataCacheFactory(dcfc);
var dc = dcf.GetDefaultCache();
const string key = "a";
var value = new int [256 * 1024]; // 1MB
for (int i = 0; i < 300; i++)
{
var putT = new Thread(() => dc.Put(key, value));
putT.Start();
var getT = new Thread(() => dc.Get(key));
getT.Start();
}
When calling Get() with different key or DataCache is synchronized, this issue will not appear. If DataCache is obtained with each call from DataCacheFactory (DataCache is supposed to be thread-safe) or timeouts are prolonged it has no effect and error is still received.
It seems to me very strange that MS would leave such bug. Did anybody faced similar issue?
I also see the same behavior and my understanding is that this is by design. The cache contains two concurrency models:
Optimistic Concurrency Model methods: Get, Put, ...
Pessimistic Concurrency Model: GetAndLock, PutAndLock, Unlock
If you use optimistic concurrency model methods like Get then you have to be ready to get DataCacheErrorCode.RetryLater and handle that appropriately - I also use a retry approach.
You might find more information at MSDN: Concurrency Models
We have seen this problem as well in our code. We solve this by overloading the Get method to catch expections and then retry the call N times before fallback to a direct request to SQL.
Here is a code that we use to get data from the cache
private static bool TryGetFromCache(string cacheKey, string region, out GetMappingValuesToCacheResult cacheResult, int counter = 0)
{
cacheResult = new GetMappingValuesToCacheResult();
try
{
// use as instead of cast, as this will return null instead of exception caused by casting.
if (_cache == null) return false;
cacheResult = _cache.Get(cacheKey, region) as GetMappingValuesToCacheResult;
return cacheResult != null;
}
catch (DataCacheException dataCacheException)
{
switch (dataCacheException.ErrorCode)
{
case DataCacheErrorCode.KeyDoesNotExist:
case DataCacheErrorCode.RegionDoesNotExist:
return false;
case DataCacheErrorCode.Timeout:
case DataCacheErrorCode.RetryLater:
if (counter > 9) return false; // we tried 10 times, so we will give up.
counter++;
Thread.Sleep(100);
return TryGetFromCache(cacheKey, region, out cacheResult, counter);
default:
EventLog.WriteEntry(EventViewerSource, "TryGetFromCache: DataCacheException caught:\n" +
dataCacheException.Message, EventLogEntryType.Error);
return false;
}
}
}
Then when we need to get something from the cache we do:
TryGetFromCache(key, region, out cachedMapping)
This allows us to use Try methods that encasulates the exceptions. If it returns false, we know thing is wrong with the cache and we can access SQL directly.