creating Pcollection from json string input - google-cloud-platform

Question

I am trying to create a unit test for my Dataflow code. However, I am getting error while creating pcollection using beam.create().
In the original function, I am using json.loads(element) which takes json string as input and gives the dictionary as o/p.
Test code:
def test(self):
input = '{"name": "xyz"}'
expected_output = {'name': 'xyz'}
input_string = p | beam.create(input) #pipeline object is already defined as p
output = input_string | beam.pardo(_splitdata) #calling the original function here
assert_that(output,equal_to(expected_output))
Errors :
beam.create() is not taking string as input. It just takes iterables and If I make input as suppose a list or any other iterable then json.loads() as part of _splitdata() expects the json object just as str.
How do I resolve this issue. Please help.

Answer 1

I got into similar scenario today, here is my solution:
import apache_beam as beam
data = [
{
'id': 1,
'name': 'abc',
},
{
'id': 2,
'name': 'xyz'
}
]
with beam.Pipeline() as pipeline:
plant_details = (
pipeline
| 'Read Input Data' >> beam.Create(data)
| beam.Map(print))
Output:
{'id': 1, 'name': 'abc'}
{'id': 2, 'name': 'xyz'}