How do I use apollo cache to match an input variable that is a sub array/sub list of ids in the cache?
In particular, when I execute an apollo query, I want to return a response for a subset of list of params.
Phrased another way, how can I add additional entries into the cache so that my cache can match future "subqueries" e.g.
fetch({
variables: {
ids: ['1']
}
})
when I had previously queried
fetch({
variables: {
ids: ['1', '2', '3']
}
})
Full example below:
// This code is not working, but just used to illustrate an example.
const [fetch, data] = useQuery( /* graphql params here */ );
// Apollo cache will cache the response for `ids: ['1', '2', '3']`
fetch({
variables: {
ids: ['1', '2', '3']
}
})
// In these subsequent call, I don't want to hit the server and want to hit the cache.
// Can I leverage apollo cache for this?
fetch({
variables: {
ids: ['1']
}
})
fetch({
variables: {
ids: ['2','3']
}
})
fetch({
variables: {
ids: ['1','2']
}
})
You should be able to use one of Key arguments or the read function for the type in question.
Related
My data structure in AWS DynamoDB looks like this:
{ key: 'roomNameOne',
value: {
attendees: ['A', 'B', 'C'] // this is a set,
wsConnections: [{ connectiondId: 'foo', domain: 'xyz.com' }, { connectiondId: 'bar', domain: 'xyz.com' }]
}
}
{ key: 'roomNameTwo',
value: {
attendees: ['X', 'Y', 'Z'],
wsConnections: [{ connectiondId: 'foo', domain: 'xyz.com' }, { connectiondId: 'bar', domain: 'xyz.com' }]
}
}
Now when I get a request that connectionId: foo is lost, I want to remove that entry from all the items.
So after DynamoDB update operation my list should look like this:
{ key: 'roomNameOne',
value: {
attendees: ['A', 'B', 'C'] // this is a set,
wsConnections: [{ connectiondId: 'bar', domain: 'xyz.com' }]
}
}
{ key: 'roomNameTwo',
value: {
attendees: ['X', 'Y', 'Z'],
wsConnections: [{ connectiondId: 'bar', domain: 'xyz.com' }]
}
}
Can you please help me with the query for update? The trick here is I don't know the room names, but while connection, I am aware of what all room names a connection is interested in.
Unfortunately, DynamoDB does not allow for this type of operation on a complex attribute (e.g. list of maps).
Modeling one-to-many relationships using complex attributes is a useful pattern. However, one of the drawbacks of this approach is that you won't be able to perform the types of operations you're describing.
If you have access patterns that require you to update wsConnections, you might consider modeling the relationship by making each entry of the wsConnections list it's own item in DynamoDB. For example
Storing your data in this way would make it easier for you to remove connections. For example, if you wanted to remove bar from your connections, you could perform the following operation
ddbClient.delete({
TableName: "YOUR_TABLE_NAME",
Key: {PK: "roomNameOne", SK: "wsConnection#bar"}
})
EDIT: If you don't have access to the PK, your only option is a scan operation.
ddbClient.scan({
"TableName": "YOUR TABLE NAME",
"FilterExpression": "contains(#key, :value)",
"ExpressionAttributeValues": {
":value": {
"S": "foo"
}
},
"ExpressionAttributeNames": {
"#key": "connections"
}
})
This will scan the entire database looking for items whose connections attribute contains "foo". This will let you fetch the list of items, which you can then update and persist back to DDB.
This approach is not ideal. The scan operation will search the entire database, which can be horribly inefficient. You'd also have to issue multiple requests to DDB; one to fetch and one to update. multiple roundtrips aren't the end of the world, but again, not ideal.
To unlock more flexible and efficient access patterns, it would be ideal to get the data out of the wsConnections list attribute. As long a the data is buried in a complex attribute, your options will be limited.
We are using apollo-client in a react project. We made a cursor level on top of any list queries. For example:
query MediaList($mediaIds: [ID!], $type: [MediaType!], $userId: ID!) {
user {
id
medias_cursor(all_medias: true, active: true, ids: $mediaIds) {
medias {
id
type
category
name
}
}
}
}
Now for different MediaList query, the Media Objects might already exist in cache but we can not use it to skip network query. For example:
After we query medias_cursor({"all_medias":true,"active":true,"ids":["361","362","363"]}),
we've already got the three Media objects here - (Media:361, Media:362, Media:363).
So when we try to query medias_cursor({"all_medias":true,"active":true,"ids":["361","363"]}, we should have everything we need in the cache already. But right now, the apollo default behavior will just pass the cache and hit the network.
We tried to add a cacheRedirects config to solve this problem like this:
const cache = new InMemoryCache({
cacheRedirects: {
User: {
medias_cursor: (_, { ids }, { getCacheKey }) => {
if (!ids) return undefined
return {
medias: map(ids, id => {
return getCacheKey({ __typename: 'Media', id: id })
})
}
},
},
},
})
We are expecting that the cacheRedirects would help us to use the cache when it's available, but now it will skip the cache anyway.
I have a fairly simple node app using AWS AppSync. I am able to run queries and mutations successfully but I've recently found that if I run a query twice I get the same response - even when I know that the back-end data has changed. In this particular case the query is backed by a lambda and in digging into it I've discovered that the query doesn't seem to be sent out on the network because the lambda is not triggered each time the query runs - just the first time. If I use the console to simulate my query then everything runs fine. If I restart my app then the first time a query runs it works fine but successive queries again just return the same value each time.
Here are some part of my code:
client.query({
query: gql`
query GetAbc($cId: String!) {
getAbc(cId: $cId) {
id
name
cs
}
}`,
options: {
fetchPolicy: 'no-cache'
},
variables: {
cid: event.cid
}
})
.then((data) => {
// same data every time
})
Edit: trying other fetch policies like network-only makes no visible difference.
Here is how I set up the client, not super clean but it seems to work:
const makeAWSAppSyncClient = (credentials) => {
return Promise.resolve(
new AWSAppSyncClient({
url: 'lalala',
region: 'us-west-2',
auth: {
type: 'AWS_IAM',
credentials: () => {
return credentials
}
},
disableOffline: true
})
)
}
getRemoteCredentials()
.then((credentials) => {
return makeAWSAppSyncClient(credentials)
})
.then((client) => {
return client.hydrated()
})
.then((client) => {
// client is good to use
})
getRemoteCredentials is a method to turn an IoT authentication into normal IAM credentials which can be used with other AWS SDKs. This is working (because I wouldn't get as far as I do if not).
My issue seems very similar to this one GraphQL Query Runs Sucessfully One Time and Fails To Run Again using Apollo and AWS AppSync; I'm running in a node environment (rather than react) but it is essentially the same issue.
I don't think this is relevant but for completeness I should mention I have tried both with and without the setup code from the docs. This appears to make no difference (except annoying logging, see below) but here it is:
global.WebSocket = require('ws')
global.window = global.window || {
setTimeout: setTimeout,
clearTimeout: clearTimeout,
WebSocket: global.WebSocket,
ArrayBuffer: global.ArrayBuffer,
addEventListener: function () { },
navigator: { onLine: true }
}
global.localStorage = {
store: {},
getItem: function (key) {
return this.store[key]
},
setItem: function (key, value) {
this.store[key] = value
},
removeItem: function (key) {
delete this.store[key]
}
};
require('es6-promise').polyfill()
require('isomorphic-fetch')
This is taken from: https://docs.aws.amazon.com/appsync/latest/devguide/building-a-client-app-javascript.html
With this code and without offlineDisabled: true in the client setup I see this line spewed continuously on the console:
redux-persist asyncLocalStorage requires a global localStorage object.
Either use a different storage backend or if this is a universal redux
application you probably should conditionally persist like so:
https://gist.github.com/rt2zz/ac9eb396793f95ff3c3b
This makes no apparent difference to this issue however.
Update: my dependencies from package.json, I have upgraded these during testing so my yarn.lock contains more recent revisions than listed here. Nevertheless: https://gist.github.com/macbutch/a319a2a7059adc3f68b9f9627598a8ca
Update #2: I have also confirmed from CloudWatch logs that the query is only being run once; I have a mutation running regularly on a timer that is successfully invoked and visible in CloudWatch. That is working as I'd expect but the query is not.
Update #3: I have debugged in to the AppSync/Apollo code and can see that my fetchPolicy is being changed to 'cache-first' in this code in apollo-client/core/QueryManager.js (comments mine):
QueryManager.prototype.fetchQuery = function (queryId, options, fetchType, fetchMoreForQueryId) {
var _this = this;
// Next line changes options.fetchPolicy to 'cache-first'
var _a = options.variables, variables = _a === void 0 ? {} : _a, _b = options.metadata, metadata = _b === void 0 ? null : _b, _c = options.fetchPolicy, fetchPolicy = _c === void 0 ? 'cache-first' : _c;
var cache = this.dataStore.getCache();
var query = cache.transformDocument(options.query);
var storeResult;
var needToFetch = fetchPolicy === 'network-only' || fetchPolicy === 'no-cache';
// needToFetch is false (because fetchPolicy is 'cache-first')
if (fetchType !== FetchType.refetch &&
fetchPolicy !== 'network-only' &&
fetchPolicy !== 'no-cache') {
// so we come through this branch
var _d = this.dataStore.getCache().diff({
query: query,
variables: variables,
returnPartialData: true,
optimistic: false,
}), complete = _d.complete, result = _d.result;
// here complete is true, result is from the cache
needToFetch = !complete || fetchPolicy === 'cache-and-network';
// needToFetch is still false
storeResult = result;
}
// skipping some stuff
...
if (shouldFetch) { // shouldFetch is still false so this doesn't execute
var networkResult = this.fetchRequest({
requestId: requestId,
queryId: queryId,
document: query,
options: options,
fetchMoreForQueryId: fetchMoreForQueryId,
}
// resolve with data from cache
return Promise.resolve({ data: storeResult });
If I use my debugger to change the value of shouldFetch to true then at least I see a network request go out and my lambda executes. I guess I need to unpack what that line that is changing my fetchPolicy is doing.
OK I found the issue. Here's an abbreviated version of the code from my question:
client.query({
query: gql`...`,
options: {
fetchPolicy: 'no-cache'
},
variables: { ... }
})
It's a little bit easier to see what is wrong here. This is what it should be:
client.query({
query: gql`...`,
fetchPolicy: 'network-only'
variables: { ... }
})
Two issues in my original:
fetchPolicy: 'no-cache' does not seem to work here (I get an empty response)
putting the fetchPolicy in an options object is unnecessary
The graphql client specifies options differently and we were switching between the two.
Set the query fetch-policy to 'network-only' when running in an AWS Lambda function.
I recommend using the overrides for WebSocket, window, and localStorage since these objects don't really apply within a Lambda function. The setup I typically use for NodeJS apps in Lambda looks like the following.
'use strict';
// CONFIG
const AppSync = {
"graphqlEndpoint": "...",
"region": "...",
"authenticationType": "...",
// auth-specific keys
};
// POLYFILLS
global.WebSocket = require('ws');
global.window = global.window || {
setTimeout: setTimeout,
clearTimeout: clearTimeout,
WebSocket: global.WebSocket,
ArrayBuffer: global.ArrayBuffer,
addEventListener: function () { },
navigator: { onLine: true }
};
global.localStorage = {
store: {},
getItem: function (key) {
return this.store[key]
},
setItem: function (key, value) {
this.store[key] = value
},
removeItem: function (key) {
delete this.store[key]
}
};
require('es6-promise').polyfill();
require('isomorphic-fetch');
// Require AppSync module
const AUTH_TYPE = require('aws-appsync/lib/link/auth-link').AUTH_TYPE;
const AWSAppSyncClient = require('aws-appsync').default;
// INIT
// Set up AppSync client
const client = new AWSAppSyncClient({
url: AppSync.graphqlEndpoint,
region: AppSync.region,
auth: {
type: AppSync.authenticationType,
apiKey: AppSync.apiKey
}
});
There are two options to enable/disable caching with AppSyncClient/ApolloClient, for each query or/and on initializing the client.
Client Config:
client = new AWSAppSyncClient(
{
url: 'https://myurl/graphql',
region: 'my-aws-region',
auth: {
type: AUTH_TYPE.AWS_MY_AUTH_TYPE,
credentials: await getMyAWSCredentialsOrToken()
},
disableOffline: true
},
{
cache: new InMemoryCache(),
defaultOptions: {
watchQuery: {
fetchPolicy: 'no-cache', // <-- HERE: check the apollo fetch policy options
errorPolicy: 'ignore'
},
query: {
fetchPolicy: 'no-cache',
errorPolicy: 'all'
}
}
}
);
Alternative: Query Option:
export default graphql(gql`query { ... }`, {
options: { fetchPolicy: 'cache-and-network' },
})(MyComponent);
Valid fetchPolicy values are:
cache-first: This is the default value where we always try reading data from your cache first. If all the data needed to fulfill your query is in the cache then that data will be returned. Apollo will only fetch from the network if a cached result is not available. This fetch policy aims to minimize the number of network requests sent when rendering your component.
cache-and-network: This fetch policy will have Apollo first trying to read data from your cache. If all the data needed to fulfill your query is in the cache then that data will be returned. However, regardless of whether or not the full data is in your cache this fetchPolicy will always execute query with the network interface unlike cache-first which will only execute your query if the query data is not in your cache. This fetch policy optimizes for users getting a quick response while also trying to keep cached data consistent with your server data at the cost of extra network requests.
network-only: This fetch policy will never return you initial data from the cache. Instead it will always make a request using your network interface to the server. This fetch policy optimizes for data consistency with the server, but at the cost of an instant response to the user when one is available.
cache-only: This fetch policy will never execute a query using your network interface. Instead it will always try reading from the cache. If the data for your query does not exist in the cache then an error will be thrown. This fetch policy allows you to only interact with data in your local client cache without making any network requests which keeps your component fast, but means your local data might not be consistent with what is on the server. If you are interested in only interacting with data in your Apollo Client cache also be sure to look at the readQuery() and readFragment() methods available to you on your ApolloClient instance.
no-cache: This fetch policy will never return your initial data from the cache. Instead it will always make a request using your network interface to the server. Unlike the network-only policy, it also will not write any data to the cache after the query completes.
Copied from: https://www.apollographql.com/docs/react/api/react-hoc/#graphql-options-for-queries
I'm going to put an item in two dynamodb tables. This is my request params of BatchWriteItem operation.
RequestItems: {
first_table: [{
PutRequest: {
Item: {
employee_id: '123',
company_id: '123',
job_position: 'manager'
}
}
}],
second_table: [{
PutRequest: {
Item: {
facility_id: '123',
company_id: '123',
job_position: 'manager'
}
}
}]
},
ReturnConsumedCapacity: "TOTAL"
My item is updated succefully but I get this response -
UnprocessedItems: {}
How can I get response with updated data? Thanks
Its not possible to get a response containing the items you have put using BatchWriteItem. PutItem can return overwritten values, but not new ones.
You might consider:
1) Using the data you already have. Afterall, you know the items have been written and you already have them.
2) If you want some statistics on your batchwrite you could use
"ReturnItemCollectionMetrics": "SIZE"
3) Query for the items after you have written them.
Let's say I have a table that lists a bunch of Posts using a query like:
const PostsQuery = gql`
query posts($name: string) {
posts {
id
name
status
}
}
`;
const query = apolloClient.watchQuery({query: PostsQuery});
query.subscribe({
next: (posts) => console.log(posts) // [ {name: "Post 1", id: '1', status: 'pending' }, { name: "Paul's Post", id: '2', status: 'pending'} ]
});
Then later my user comes along and enters a value in a search field and calls this code:
query.setVariables({name: 'Paul'})
It fetches the filtered posts and logs it out fine.
// [ { name: "Paul's Post", id: '2', status: 'pending'} ]
Now, in my table there is a button that changes the status of a post from 'Pending' to 'Active'. The user clicks that and it calls code like:
const PostsMutation = gql`
mutation activatePost($id: ID!) {
activatePost(id: $id) {
ok
object {
id
name
status
}
}
}
`;
apolloClient.mutate({mutation: PostsMutation});
All is well with the mutation, but now I want to refetch the table data so it has the latest, so I make a change:
apolloClient.mutate({
mutation: PostsMutation,
refetchQueries: [{query: PostsQuery, variables: {name: 'Paul'}]
});
Hurray, it works!
// [ { name: "Paul's Post", id: '2', status: 'active'} ]
But... now my user clears the search query, expecting the results to update.
query.setVariables({});
// [ {name: "Post 1", id: '1', status: 'pending' }, { name: "Paul's Post", id: '2', status: 'pending'} ]
Oh no! Because the data was not refetched in our mutation with our "original" variables (meaning none), we are getting stale data!
So how do you handle a situation where you have a mutation that may affect a query that could have many permutations of variables?
I had a similar issue, I am using Apollo with Angular, so I am not sure if this method will work with React Client, but it should.
If you look closely at refetchQueries properties on the mutate method, you will see that the function can also return a string array of query names to refetch. By returning just the query name as a string, you do not need to worry about the variables. Be advised that this will refetch all the queries matching the name. So if you had a lot queries with different variables it could end up being a large request. But, in my case it is worth the trade off. If this is a problem, you could also get access to the queryManager through apolloClient.queryManager which you could use to do some more fine grained control of what to refetch. I didn't implement it, but it looked very possible. I found the solution below fits my needs fine.
In your code, what you need to do is:
apolloClient.mutate({
mutation: PostsMutation,
refetchQueries: (mutationResult) => ['PostQueries']
});
This will refetch any query with the name 'PostQueries'. Again, it is possible to only refetch a subset of them if you dig into the queryManager and do some filtering on the active watch queries. But, that is another exercise.