How to prepare Pub/Sub emulator for tests? - google-cloud-platform

Question

I start the gcloud sdk docker
docker run -ti --rm --expose=8085 -p 8085:8085 google/cloud-sdk:latest
then i run:
gcloud beta emulators pubsub start --project=my-project --host-port=0.0.0.0:8085
then stop the sever and then:
gcloud beta emulators pubsub env-init
gives:
export PUBSUB_EMULATOR_HOST=0.0.0.0:8085
but there is no project id. How can I setup project for tests? How can i create topics and subscriptions?
version:
gcloud version
gives:
Google Cloud SDK 236.0.0
...
pubsub-emulator 2019.02.22

Answer 1

You are launching pubsub emulator with project my-project in your 2nd command. Once this is running, don't kill it, leave it running.
To create the topics and subscriptions, you have to use one of the SDKs. I created a demo project that does this using the Java SDK: https://github.com/nhartner/pubsub-emulator-demo/
The relevant code is this:
#Component
public class TestPubSubConfig {
private final TransportChannelProvider channelProvider;
private final CredentialsProvider credentialsProvider;
private String projectId;
private String topicName = "test-topic";
private String subscriptionName = "test-subscription";
TestPubSubConfig(#Autowired #Value("${spring.cloud.gcp.pubsub.emulator-host}") String emulatorHost,
#Autowired #Value("${spring.cloud.gcp.project-id}") String projectId) throws IOException {
this.projectId = projectId;
ManagedChannel channel = ManagedChannelBuilder.forTarget(emulatorHost).usePlaintext().build();
channelProvider = FixedTransportChannelProvider.create(GrpcTransportChannel.create(channel));
credentialsProvider = NoCredentialsProvider.create();
createTopic(topicName);
createSubscription(topicName, subscriptionName);
}
#Bean
public Publisher testPublisher() throws IOException {
return Publisher.newBuilder(ProjectTopicName.of(projectId, topicName))
.setChannelProvider(channelProvider)
.setCredentialsProvider(credentialsProvider)
.build();
}
private void createSubscription(String topicName, String subscriptionName) throws IOException {
ProjectTopicName topic = ProjectTopicName.of(projectId, topicName);
ProjectSubscriptionName subscription = ProjectSubscriptionName.of(projectId, subscriptionName);
try {
subscriptionAdminClient()
.createSubscription(subscription, topic, PushConfig.getDefaultInstance(), 100);
}
catch (AlreadyExistsException e) {
// this is fine, already created
}
}
private void createTopic(String topicName) throws IOException {
ProjectTopicName topic = ProjectTopicName.of(projectId, topicName);
try {
topicAdminClient().createTopic(topic);
}
catch (AlreadyExistsException e) {
// this is fine, already created
}
}
private TopicAdminClient topicAdminClient() throws IOException {
return TopicAdminClient.create(
TopicAdminSettings.newBuilder()
.setTransportChannelProvider(channelProvider)
.setCredentialsProvider(credentialsProvider).build());
}
private SubscriptionAdminClient subscriptionAdminClient() throws IOException {
return SubscriptionAdminClient.create(SubscriptionAdminSettings.newBuilder()
.setTransportChannelProvider(channelProvider)
.setCredentialsProvider(credentialsProvider)
.build());
}
}

Answer 2

A possible gotchya we uncovered while working with the Pub/Sub emulator is that the documentation says:
In this case, the project ID can be any valid string; it does not
need to represent a real GCP project because the Cloud Pub/Sub
emulator runs locally.
any valid string in this context is not any string, but specifically a valid one, meaning it looks like a valid GC Project Id. In our testing this was specifically strings that match the REGEX pattern:
/^[a-z]-[a-z]-\d{6}$/
Once supplied with a valid project ID, the emulator works as advertised. If you have a sandbox project in GC you can use that ID or you can make up your own that matches that pattern. Once you got that far you can follow the remainder of the Testing apps locally with the emulator documentation.