I am working on an contact sync solution to be able to keep the contacts in our app in sync with the google contacts of the user.
Our code uses the php library for the google people api latest Version (v1).
Everything is working fine for one week with each user but after that week we get:
400 - Error "Sync token is expired.
Clear local cache and retry call without the sync token".
My question now:
Is this intended behaviour that you have to clear all your cache after one week with no changes or am I doing something wrong?
Is there any possibility to renew a syncToken if there were no changes?
I already checked the whole code to be sure that the new received nextSyncToken is saved at our side and used for the next incremental sync request. It seems that the new sync Token is always the same as the one sent in the request. Thus it is just clear that we get that errors if a sync token expires after exactly one week.
I also tried to set the option requestSyncToken to true for every list request, even if also a syncToken is set. No success. Sync token stays the same after each request with no changes.
Just in case someone is also facing this problem (syncToken expiration after one week without changes in the persons/contacts list):
Our solution was:
Save the creation date and time of a new syncToken each time you
get one together with the syncToken.
When you receive a syncToken in an incremental sync process compare that token to the stored one. If the syncToken is a new
one, overwrite the old one and its creation date/time.
Use a continuous process that checks each syncToken. If one is about one week old (for security reasons we used 6 days) create a new
syncToken (process see below). As the people API does not offer
things like the watch-channels of the calendar API you would anyway
need some continuous processes that do list-calls in fixed time
intervals for a complete real-time synchronization - so maybe you
could combine these tasks depending on your solution for this
problem?!
Process for creation of a new SyncToken:
Do a new list request without providing a syncToken.
For additional security do some checkups like compare the total persons received with the total persons expected by the old/current
data. And do this renew process at a time of the day when almost no
one does changes generally, for example like 2am.
Overwrite the old syncToken and date/time with the new one and the current date/time.
That's it.
But attention! You can still miss some changes that were made if your syncToken renew process is running exactly at the time a change is made!
Create a dummy contact before the sync to get a new syncToken. After the sync delete the dummy contact from both Google Contacts and your cache.
Related
After my users log in the app makes too many requests to DynamoDB and I am thinking about different ways to reduce the number of calls.
The app allows user to trigger certain alerts that get sent to other users. For instance: "Shipment received, come to the deck", "Shipment completed", etc.
These are the calls made:
Get company's software license expiration date.
Get the computer's location in the building (i.e. "Office A").
Get the kinds of alerts that can be triggered (i.e. "Shipment received, come to the deck", "Shipment completed", etc).
Get information about the user (i.e. company teams the user belongs to, and admin level the user has (which can be 0, 1, 2, or 3).
Potential solutions I have though about:
Put the company's license expiration date as an attribute of each computer (This would reduce the number of queries by 1). However, if I need to update the company's license expiration date, then I need to update it for EVERY SINGLE computer I have in the system, which sounds impractical to me since I may have 200, 300 or perhaps even more computers in the database.
Add the company's license expiration date as an attribute of the alerts (This would reduce the number of queries by 1); which seems more reasonable because there are only about 15 different kinds of alerts, so if I need to change the license expiration date later on, it is not too bad.
Cache information on the user's device; however, I can't seem to find a good strategy to keep the information stored locally as updated as possible.
I still think these 3 options do not sound too good, so I am hoping someone can point me in the right direction. Is there a good way to reduce the number of calls? I am retrieving information about 4 different entities (license, computer, alert, user), should I leave those 4 calls after users log in?
here are few things that can be done wrt each component.
Get information about the user
keep it in session store and whenever details changes update the store. session stores are usually implemented using cache like redis.
Computer location
Keep it in a distributed cache like redis. lazily initialise it. and whenever new write happens to computer location (rare IMO) remove the entry from redis using dynamodb streams and aws lambda.
Kind of alerts
Same as Computer location
License expiration date
If possible don't allow license expiry date (issue a new one for these cases, so that traceability is maintained.) and cache licence expiry forever. OR same as Computer location.
I am setting up Sitecore xDB and am trying to test exactly what info gets through the system for authenticated and non-authenticated users. I would like to be able to make a change and see the results quickly in Sitecore. I found the setting to lower session lifetime to 1 minute rather than 20. I have not found a way to just force Sitecore to sync with Mongo on demand or at least within 1-5 minutes rather than, what also appears to be about 20 minutes at the moment. Does it exist or is "rebuilding" the database explained here the only existing process?
See this blog post by Martina Welander for this and more good info about xDB sessions: https://mhwelander.net/2016/08/24/whats-in-a-session-what-exactly-happens-during-a-session-and-how-does-the-xdb-know-who-you-are/
You just need a utility page that calls System.Web.HttpContext.Current.Session.Abandon(). You may also want to redirect the user to a page that doesn't exist.
Update to address comment
My understanding is that once an xDB session has expired, processing should take place quickly. In the Sitecore.Analytics.Processing.Services.config file, the BackgroundService agent is set to run on an interval of 15 seconds by default.
You may just be seeing cached reporting data. Try clearing the cache using the /sitecore/admin/cache.aspx page. You could also decrease the defaultCacheExpiration setting for the reporting cacheProvider in the Sitecore.Analytics.Reporting.config file. The default is 10 minutes.
We have sysdig running on our WSO2 API gateway machine and we notice that it fires a large number of SQL queries to the database for a minute, than waits a minute and repeats.
The query looks like this:
Every minute it goes wild, waits for a minute and goes wild again with a request of the following format:
SELECT REG_PATH, REG_USER_ID, REG_LOGGED_TIME, REG_ACTION, REG_ACTION_DATA
FROM REG_LOG
WHERE REG_LOGGED_TIME>'2016-02-29 09:57:54'
AND REG_LOGGED_TIME<'2016-03-02 11:43:59.959' AND REG_TENANT_ID=-1234
There is no load on the server. What is causing this? What can we do to avoid this?
screen shot sysdig api gateway process
This particular query is the result of the registry indexing task that runs in the background. The REG_LOG table is being queried periodically to retrieve the latest registry actions. The indexing task cannot be stopped. However, one can configure the frequency of the indexing task through the following parameter that is in the registry.xml. See [1] for more information.
indexingFrequencyInSeconds
If this table is filled up, one can clean the data using a simple SQL query. However, when deleting the records, one must be careful not to delete all the data. The latest records of each resource path should be left in the REG_LOG table since reindexing of data requires at least one reference of each resource path.
Also, if required, before clearing up the REG_LOG table, you can take a dump of the data in case you do not want to loose old records. Hope this answer provides information you require.
[1] - https://docs.wso2.com/display/Governance510/Configuration+for+Indexing
I would like to use the list method in the Reports API to periodically fetch Activities of all users of some applications (e.g. 'admin' and 'login') and keep a local copy of all that data (using watch and push notifications is not an option in my particular scenario).
The idea is defining small time windows (e.g. 60 seconds) and, at the end of each time window plus some small delay, using the 'list' method and setting the startTime and endTime accordingly, fetching all events logged during the already finished time window.
This way I would be able to have an almost-real-time list of events locally stored. However, I'm not sure about what minimal delay should be used to ensure that the list method will be able to fetch all events. I'm assuming some delay is required here. Am I right? If so, is there any minimum delay that guarantees all events will be fetched?
In theory you wouldn't need a delay, but probably 10 secs would be fine if you want to be sure. Another important thing would be the api quota, in this case the project would be limited to 5 queries per second. Here is the documentation on that https://developers.google.com/admin-sdk/reports/v1/limits
I need to track some information on users, but would like to retain it for a fixed time period, say a week.
If I set this value via request.sessions, and the user logs out, can I retrieve it if they log back in later? This all assumes that my sessions are normally set to expire in 30 days, if the user neVer logs out.
While thinking about the above problem, I decided to store the data in a table, but I would still like to know the answer to above for referenCe. I also decided not to use cookies due to unreliability.
It would depend on your session backend. But the default backend (backends.db) does delete the row from the sessions table when you log out.
I would recommend adding the data to a field in the user profile. Using the session will give problems even if you don't delete the data. The next time the user logs in you won't know which session id he/she used the last time and normally you only have the session id to look up. Not a user id so you can get all sessions owned by a specific user.