I am using Azure EventHub and capturing the contents to blob storage using https://learn.microsoft.com/en-us/azure/event-hubs/event-hubs-capture-overview.
Now for the generated Avro files, why is the information stored in the properties and System properties fields empty?
NOTE Azure is populating these fields
I publish the data using a POST request with my payload as the body and authorization headers set.
Am I missing additional headers which would be required to make Azure fill these columns?
edit
so the event hub client`s POST method looks like this:
private val SB_URL = "https://$namespace.servicebus.windows.net/$eventHub"
private val HEADER_AUTHORIZATION = AzureUtil.getSasToken(SB_URL, sas_key_name, sas_key)
private val HEADER_CONTENT_TYPE = "application/atom+xml;type=entry;charset=utf-8"
private val REQ_URL = "https://$namespace.servicebus.windows.net/$eventHub/messages"
private val REQ_TIMEOUT = "60"
private val REQ_API_VERSION = "2014-01"
private val client = OkHttpClient()
private val JSON = "application/json; charset=utf-8".toMediaType()
private var callback: Callback? = null
fun registerCallback(cb: Callback) {
callback = cb
}
fun send(message: String) {
val request = Request.Builder()
.url("$REQ_URL?timeout=$REQ_TIMEOUT&api-version=$REQ_API_VERSION")
.addHeader("Content-Type", HEADER_CONTENT_TYPE)
.addHeader("Authorization", HEADER_AUTHORIZATION)
.post(message.toRequestBody(JSON))
.build()
val call = client.newCall(request)
try {
val response = call.execute()
callback!!.onResponse(call, response)
} catch (error: IOException) {
callback!!.onFailure(call, error)
}
}
Related
I've been trying to use a custom authorizer ( lambda authorizer ) of type Request for a certain HTTP API in Kotlin.
Taking inspiration from this, I wrote the following :
class AuthorizerHandler : RequestHandler<Map<String, Any>, AuthorizerResponse>{
override fun handleRequest(input: Map<String, Any>, context: Context?): AuthorizerResponse {
val LOG = LogManager.getLogger(AuthorizerHandler::class.java)
LOG.info(input)
val headers = input["headers"]!! as Map<*, *>
val authorization = headers["authorization"]!! as String
val arn = input["routeArn"]!! as String
val tokenBody: TokenBody = TokenBody.fromToken(authorization)
LOG.info("SUB : ${tokenBody.userID}")
val proxyContext = input["requestContext"] as Map<*, *>
val principalId = proxyContext["accountId"] as String
val statement = IamPolicyResponse.Statement.builder().withResource(listOf("*"))
.withEffect("Allow")
.withAction("execute-api:Invoke")
.build()
val policyDocument = IamPolicyResponse.PolicyDocument.builder()
.withStatement(listOf(statement))
.withVersion("2012-10-17")
.build()
return AuthorizerResponse(principalId = tokenBody.userID, policyDocument = policyDocument)
}
}
TokenBody is custom class with init function fromToken that helps me deconstruct the token. It's working as expected and tokenBody.userID gives me the user's sub.
AuthorizerResponse is also a custom class which looks like this :
class AuthorizerResponse (
#JsonProperty("principalId")
val principalId: String? = null,
#JsonProperty("policyDocument")
val policyDocument: IamPolicyResponse.PolicyDocument? = null,
#JsonProperty("context")
val context: Map<String, String>? = null,
)
First I was supplying arn as the Resource when building my statement. Calling the API resulted in
{
"message": "Internal Server Error"
}
Changing Resource to * doesn't help either and throws the same 500 Internal Server Error.
Similarly, I've tried accountID as principalID as well but that doesn't work either.
I've checked the logs of the Authorizer lambda but that doesn't throw any error so it's technically working as expected after getting invoked properly.
Any ideas on how this can be fixed?
Edit : Mentioned the type of API.
I am mocking HttpClient to mock it's post request to an API endpoint. Here's the test case code :-
class SomeClassTest {
#Mock
lateinit var someClassObj: SomeClass
#Mock
var httpClient = HttpClient(CIO)
// Read config properties
private val testEnv = createTestEnvironment {
config = HoconApplicationConfig(ConfigFactory.load("application.conf"))
}
private fun getMockedHttpClientCall(customResponse: String): HttpClientCall {
// Return HttpClientCall object with custom response and status = OK
val httpClientCall = mockk<HttpClientCall> {
every { client } returns mockk {}
coEvery { receive(io.ktor.util.reflect.typeInfo<String>()) } returns customResponse
every { coroutineContext } returns EmptyCoroutineContext
every { attributes } returns Attributes()
every { request } returns object : HttpRequest {
override val call: HttpClientCall = this#mockk
override val attributes: Attributes = Attributes()
override val content: OutgoingContent = object : OutgoingContent.NoContent() {}
override val headers: Headers = Headers.Empty
override val method: HttpMethod = HttpMethod.Get
override val url: Url = Url("/")
}
every { response } returns object : HttpResponse() {
override val call: HttpClientCall = this#mockk
override val content: ByteReadChannel = ByteReadChannel(customResponse)
override val coroutineContext: CoroutineContext = EmptyCoroutineContext
override val headers: Headers = Headers.Empty
override val requestTime: GMTDate = GMTDate.START
override val responseTime: GMTDate = GMTDate.START
override val status: HttpStatusCode = HttpStatusCode.OK
override val version: HttpProtocolVersion = HttpProtocolVersion.HTTP_1_1
}
}
return httpClientCall
}
#Before
fun setUp() {
MockitoAnnotations.openMocks(HttpClient::class)
MockitoAnnotations.openMocks(SomeClass::class)
someClassObj = SomeClass(httpClient)
}
#Test
fun Test() = withApplication(testEnv) {
runBlocking {
val mockedEngine = MockEngine { request ->
respond(
content = "{\"response\":\"some sample response\"}",
status = HttpStatusCode.OK,
headers = headersOf(HttpHeaders.ContentType, "application/json")
)
}
this#SomeClassTest.httpClient = HttpClient(engine = mockedEngine)
someClassObj.setHttpClient(httpClient)
Mockito.`when`(httpClient.post<HttpResponse>(Configuration.properties.getAPIEndpoint.toString())).thenReturn(getMockedHttpClientCall("{\"response\":\"some sample response\"}").response)
assertEquals("{\"response\":\"some sample response\"}", someClassObj.func())
}
}
}
The class "SomeClass" is dedicated to call external APIs. It takes an HttpClient object in its constructor and has a setter method setHttpClient(). I am using MockEngine to mock the HttpClient.
The "someClassObj.func()" function makes a post request to the API endpoint.
The API endpoint is stored in the configuration file which I am reading in the Mockito stubbing statement.
The function "getMockedHttpClientCall()" takes a custom response as String, and uses MockK to return an "HttpClientCall" object with its HttpResponse = customResponse. This is used in the Mockito stubbing statement (in the thenReturn part). I have mocked the entire HttpClientCall because I couldn't find any other way to mock only the HttpResponse.
When I run the test case, I get an error saying io.ktor.client.features.ClientRequestException: Client request(<the api endpoint>) invalid: 404 Not Found. Text: "". However, when I make the same POST request in postman, I am getting the desired response from the API. Any idea why this may be happening?
Can not pass a list of strings to a Web API endpoint. Why?
Here is my controller:
[Route("[controller]")]
[ApiController]
public class MyController
{
[HttpPost("foo")]
public string MyMethod(List<string> strs)
{
return "foo";
}
}
Here is how I am trying to call it:
var strs = new List<string> { "bar" };
var json = JsonConvert.SerializeObject(strs);
var content = new StringContent(json, Encoding.UTF8, "application/json");
var response = await httpCliet.PostAsync("/My/foo", content);
Before calling the endpoint I place a breakpoint on the return "foo"; line. Once the breakpoint is hit the strs list inside the MyController.MyMethod is empty. The strs is not null, but it contains no elements. While my intentions and expectations are to see the strs containing one element, i.e. the string "bar".
I am using the ASP.NET Core 2.2 in project where I create and use the HttpClient. And I am using the same ASP.NET Core 2.2 in project where I have the endpoint.
I am not sure what is wrong here. I have checked a few sources. E.g. the following:
C# HTTP post , how to post with List<XX> parameter?
https://carldesouza.com/httpclient-getasync-postasync-sendasync-c/
https://blog.jayway.com/2012/03/13/httpclient-makes-get-and-post-very-simple/
And I can not find what I am missing according to those resources.
UPDATE
The following call works for me as expected:
var json = JsonConvert.SerializeObject(string.Empty);
var content = new StringContent(json, Encoding.UTF8, "application/json");
var response = await server.CreateClient().PostAsync("/My/foo?strs=bar", content);
Maybe someone knows why the parameters in my case are read from the query string only, but not from body?
You can change your url to a full url in client.PostAsync.Here is a demo worked:
Api(localhost:44379):
WeatherForecastController:
[HttpPost("foo")]
public string MyMethod(List<string> strs)
{
return "foo";
}
Call(localhost:44326):
public async Task<IActionResult> CheckAsync() {
HttpClient client = new HttpClient();
var strs = new List<string> { "bar","bar1","bar2" };
var json = JsonConvert.SerializeObject(strs);
var content = new StringContent(json, Encoding.UTF8, "application/json");
var response = await client.PostAsync("https://localhost:44379/WeatherForecast/foo", content);
return Ok(response);
}
result:
Is it possible to pass a specific value into an extension function with receiver, in a unit test?
I'm trying to test the folowing subscribe method:
...
private lateinit var subscription: SubscriptionReceiveChannel<UiStateModel>
suspend fun subscribe(model: MainViewModel) {
subscription = model.connect()
subscription.consumeEach { value -> loadView(value) /** or loadView(it) */ }
}
private fun loadView(uiState: UiStateModel) {
when(uiState) {
is Loading -> view.isLoading()
is Error -> view.isError(uiState.exception.localizedMessage)
is Success -> when {
uiState.result != null -> view.isSuccess(uiState.result)
else -> view.isEmpty()
}
}
}
I want to be able to apply a specific value to the consumeEach function, but how can this be done?
Here's my unit test:
...
private val view = mock<MainView>()
private val model = mock<MainViewModel>()
private val subscription = mock<SubscriptionReceiveChannel<UiStateModel>>()
private val presenter = MainPresenter(view)
#Test
fun `When uistate is loading, view should show loading message`() = runBlocking {
// Given
val state = UiStateModel.Loading()
whenever(model.connect()).thenReturn(subscription)
whenever(subscription.consumeEach { any() }).thenAnswer({ state })
// When
presenter.subscribe(model)
// Then
verify(model).connect()
verify(view).isLoading()
verify(view, never()).isSuccess(anyString())
verify(view, never()).isEmpty()
verify(view, never()).isError(anyString())
}
...
I am querying a search api and I need to add the query params in json as body to a post request.
val headers: scala.collection.immutable.Seq[HttpHeader] = scala.collection.immutable.Seq(
RawHeader("accept", "application/json"),
RawHeader("authorization", "xxxxxxxxxxxxxxxxxxxx"),
RawHeader("content-type", "application/json"),
RawHeader("x-customer-id", "123456789")
)
val formData = FormData(Map(
"Keywords" -> "query", "Count" -> "25"
))
val request = HttpRequest(HttpMethods.POST, "https://api.xxx.com/services/xxx/v1/search?client_id=xxxxxx", headers, formData.toEntity)
Will using formData.toEntity send it as json in body of the post?
I created a class for search query and serialized it and created an HttpEntity like so:
case class SearchObject(keyWords: String, count: Int)
val reqHeaders: scala.collection.immutable.Seq[HttpHeader] = scala.collection.immutable.Seq(
RawHeader("accept", "application/json"),
RawHeader("authorization", "xxxxxxxxxxxxxxxxxxxx"),
RawHeader("content-type", "application/json"),
RawHeader("x-customer-id", "123456789")
)
val searchObject = net.liftweb.json.Serialization.write(req) //req is search object
val searchObjectEntity = HttpEntity(ContentTypes.`application/json`, searchObject)
val request = HttpRequest(HttpMethods.POST, "https://api.xxxxxxxx.com/services/xxxxxx/v1/search?client_id=45854689", reqHeaders, searchObjectEntity)