Test case for destructuring in clojure for database connectivity - clojure

I started working on clojure a week back and i am not able to write a test case for the below code for destructuring of maps using default values:
(defn connect-db [{:keys [host port db-name username password]
:or {host "localhost"
port 12345
db-name "my-db"
username "db-user"
password "secret"}
:as cfg}]
(if (= " " host)
"Error: host name not available"
(try
(println "connecting to:" host "port:" port "db-name:" db-name
"username:" username "password:" password)
(catch Exception e(str "caught exception:" (.getMessage e))))))
The test case should be written using deftest.

I'm not sure if there is much to gain from testing destructuring, because it's a core part of the language and is built into fn, defn, let etc. Assume that it works and test only your own code. If you want to build confidence that it works the way you expect, play about with it in the REPL. There are plenty of examples here.
We can try out some defaults like this:
(let [{:keys [a b]
:or {a "a" b "b"}}
nil]
[a b])
=> ["a" "b"]

(deftest con-db
(is (= true (map? db)))
(is (= true (function? connectdb)))
(connectdb ["01" "20" "02" "03" "04"] {:host "localhost" :port 8888 :db-name "dbname" :username "name" :password "admin"})
(is(.equals "connecting: remote port: 8888 db-name: dbname username: name password: admin" outputdb))
(is(not(.equals "string" outputdb)))
(is(.equals "01-20" output))
(is(= 400 (- y x)))))

Related

Google Indexing Api in clojure

(:require [utils.base64 :as base64])
(:import [com.google.api.client.googleapis.auth.oauth2 GoogleCredential]
[com.google.api.services.indexing.v3 Indexing]
[com.google.api.services.indexing.v3.model UrlNotification]
[com.google.api.client.http HttpTransport]
[com.google.api.client.json.gson GsonFactory]
[java.io ByteArrayInputStream]
[java.util Base64]
[com.google.api.services.indexing.v3 IndexingScopes]))
;; Set the private key and client email
(def private-key "")
(def client-email "my-client-email")
(def encoded-private-key (base64/encode-string private-key))
;; Set the private key and client email
(def private-key-json '{
"type": "service_account",
"client_email": "' client-email '",
"private_key": "' encoded-private-key '"
}')
;; Create the input stream from the JSON string
(def input-stream (ByteArrayInputStream. (.getBytes private-key-json)))
;; Create the OAuth 2.0 credentials
(def credentials (doto (GoogleCredential/fromStream input-stream)
(.createScoped (java.util.Collections/singleton IndexingScopes/INDEXING))))
;; Set up the HTTP transport and JSON factory
(def http-transport (HttpTransport/newTrustedTransport))
(def json-factory (GsonFactory/getDefaultInstance))
;; Set up the Indexing API client
(def indexing-client (doto (Indexing/Builder. http-transport json-factory credentials)
(.setApplicationName "My Indexing App")
(.build)))
;; Publish a URL notification
(def url-notification (UrlNotification. "https://aaa.com" "URL_UPDATED"))
(.execute (indexing-client/urlNotifications) (publish url-notification))
I'm trying to use private-key-json but the format is not valid. What is the best way to pass data? I referred https://developers.google.com/search/apis/indexing-api/v3/prereqs#examples.
Here I'm trying not to upload json file which we get once we create a service account and make use of only mandatory fields like client_email, private_key and type fields.
Modified code:
(:require [cheshire.core :as json]
[sketches.utils.base64 :as base64]
[clojure.string :as str])
(:import [com.google.api.client.googleapis.auth.oauth2 GoogleCredential]
[com.google.api.services.indexing.v3 Indexing$Builder]
[com.google.api.services.indexing.v3.model UrlNotification]
[com.google.api.client.googleapis.javanet GoogleNetHttpTransport]
[com.google.api.client.http HttpTransport]
[com.google.api.client.json.gson GsonFactory]
[java.io ByteArrayInputStream]
[com.google.api.services.indexing.v3 IndexingScopes]))
(def creds-json {:type "service_account",
:client_email "test-indexing-api",
:client_id "117578194507835125202",
:private_key_id "964321e5fc29980944da518887116ea50dfb7803",
:private_key ""})
(defn string->stream
([s] (string->stream s "UTF-8"))
([s encoding]
(-> s
(.getBytes encoding)
(ByteArrayInputStream.))))
(defn authorize [cred]
(-> cred
(json/encode)
(string->stream)
(GoogleCredential/fromStream)
(.createScoped [(IndexingScopes/INDEXING)])))
(defn valid-private-key [private-key]
(try
(when private-key
(->> (str/trim private-key)
(re-matches #"^-----BEGIN PRIVATE KEY-----[\s|\S]*-----END PRIVATE KEY-----[\\|n]*")
(some?)))
(catch Exception e
false)))
(def encoded-private-key (base64/encode-string private-key))
(def private-creds-json-string (json/generate-string creds-json))
(def input-stream (ByteArrayInputStream. (.getBytes private-creds-json-string)))
(defn build-analytics-client [creds]
(-> (Indexing$Builder. (GoogleNetHttpTransport/newTrustedTransport) (GsonFactory/getDefaultInstance) (authorize creds))
(.build)))
(defn url-notification []
(-> (UrlNotification.)
(.setType "URL_update")
(.setUrl "http://ace.madrid.quintype.io")))
(defn realtime-data [creds]
(-> (build-analytics-client creds)
(.urlNotifications)
(.publish (url-notification))
(.execute)))
```
But I get this error `Execution error (NoSuchMethodError) at com.google.api.client.http.ConsumingInputStream/close (ConsumingInputStream.java:40).
com.google.common.io.ByteStreams.exhaust(Ljava/io/InputStream;)J`

next.jdbc - unable to find valid certification path to requested target

The error
; Execution error (SunCertPathBuilderException) at
sun.security.provider.certpath.SunCertPathBuilder/build
(SunCertPathBuilder.java:141). ; unable to find valid certification
path to requested target
The code (Clojure)
(ns backend.core
(:require [next.jdbc :as jdbc]))
(defn -main
"I don't do a whole lot ... yet."
[& args]
(println "Hello, World!"))
(def db {:dbtype "sqlserver" :dbname "dbname" :user "MY_USER" :password "MY_PASSWORD" :host "MY_HOST" :port 1234})
(def datasource (jdbc/get-datasource db))
(defn create-product
"Create a product."
[name ds]
(jdbc/execute! ds [(str "insert into dbo.product (name) value('" name "')")]))
(comment
(create-product "my-product" datasource))
I'm playing around with clojure/sql server/next.jdbc and trying to work with my distant SQL Server 2017, but this error appear...
It look like I need some certificate. Is it the case? How can I generate it? How can I install it on my dev PC? Should it be install in a specific place?
I fixed my issue by changing the content of the (def db ...). Here is the updated code working.
(ns backend.core
(:require [next.jdbc :as jdbc]))
(defn -main
"I don't do a whole lot ... yet."
[& args]
(println "Hello, World!"))
(def db {:jdbcUrl "jdbc:sqlserver://MY_DISTANT_HOST:12345;databaseName=DB_NAME;user=USERNAME;password=MY_PASSWORD;encrypt=true;trustServerCertificate=true"})
(def datasource (jdbc/get-datasource db))
(defn create-product
"Create a product."
[name ds]
(with-open [connection (jdbc/get-connection ds)]
(jdbc/execute! connection ["insert into dbo.product (name) values (?)" name] {:return-keys true})))
(comment
(create-product "my-product" datasource))
You can find more information about jdbc connection string here: https://learn.microsoft.com/en-us/sql/connect/jdbc/building-the-connection-url?view=sql-server-ver15
I too ran into a second issue after updating the data-source configuration. The error was:
no mssql-jdbc_auth-8.2.1.x64 in java.library.path
I only needed to download zip file here https://learn.microsoft.com/en-us/sql/connect/jdbc/download-microsoft-jdbc-driver-for-sql-server?view=sql-server-ver15, unzip it and follow instructions inside install.txt and finally restart my IDE (VS Code).

Why can't I extract with-redefs into a seperate function?

I have a function that I would like to test.
(defn -main
[& args]
(let [port (Integer/parseInt (or (first args) "8080"))]
(run-jetty
app
{:port port})))
I need to mock away run-jetty for obvious reasons, so I use with-redefs to verify that the correct args are passed. This works great.
(it "should overide the port with first arg"
(with-redefs [run-jetty (fn [handler opts] [handler opts])]
(should= 8081 (jetty-port (underTest/-main "8081")))))
I'd like to extract the with-redefs call since I use it in a few more tests, so I tried this:
(defn mock-jetty
[test]
(with-redefs [run-jetty (fn [handler args] [handler args])] test))
(describe "The app"
(it "should default to port 8080"
(mock-jetty
(should= 8080 (jetty-port (underTest/-main))))))
This test calls the real run-jetty function and starts a server. Why does this happen?
EDIT
Changing extraction to a macro:
(defmacro mock-jetty
[someTest]
`(with-redefs [run-jetty (fn [handler args] [handler args])] ~someTest))
(describe "The app"
(it "should default to port 8080"
(mock-jetty
(should= 8080 (jetty-port (underTest/-main))))))
throws this exception:
Can't use qualified name as parameter: boost-bin.handler-test/handler
I can guess, that it is connected with the fact that with-redefs is a macro. So when you call it straight in code, it executes test body in context of redefs, while when you move it out to the function, the body (should= 8080 (jetty-port (underTest/-main))) is being executed before being passed to macro (as it always happens to functions' params)
To make it work, you could rewrite mock-jetty as a macro:
(defmacro mock-jetty
[test]
`(with-redefs [run-jetty (fn [handler# args#] [handler# args#])] ~test))
I'm pretty sure it would help

Clojure - use a core.async channel with Yada/Aleph

I am trying to use Clojure manifold library, and in order to understand it, I need wanted to convert a core.async channel into a manifold stream.
I would like to create the equivalent the following using a core.async channel :
(require '[manifold.stream :as s])
(s/periodically 100 #(str " ok "))
;; Here is what I tried, it fails with an error 500
(let [ch (chan)]
(go-loop []
(>! ch " ok ")
(<! (timeout 100))
(recur))
(s/->source ch))
I am trying to feed a core.async channel into yada. The first code sample, using manifold.stream/periodic works, not the others using core.async. I tried on yada 1.0.0 and 1.1.0-SNAPSHOT.
Using manifold.stream/periodic works :
(def get-stream
(yada (fn [ctx]
(-> (:response ctx)
(assoc :status 202)
(assoc :body (s/periodically 1000 #(str (System/currentTimeMillis) " ")))))
{:representations [{:media-type "application/json"
:charset "UTF-8"}
{:media-type "application/edn"
:charset "UTF-8"}]}))
Using manifold.stream/->source returns an error 500 :
(def get-stream
(yada (fn [ctx]
(-> (:response ctx)
(assoc :status 202)
;; Similar to this : https://github.com/juxt/yada/blob/94f3ee93de155a8513b27e0508608691ed556a55/dev/src/yada/dev/async.clj
(assoc :body (let [ch (chan)]
(go-loop []
(>! ch " ok ")
(<! (timeout 100))
(recur))
(s/->source ch)))))
{:representations [{:media-type "application/json"
:charset "UTF-8"}
{:media-type "application/edn"
:charset "UTF-8"}]}))
;; Error on the page :
;; 500: Unknown
;; Error on GET
;; #error {
;; :cause "No implementation of method: :to-body of protocol: #'yada.body/MessageBody found for class: manifold.stream.async.CoreAsyncSource"
;; :via
;; [{:type clojure.lang.ExceptionInfo
;; :message "Error on GET"
;; :data {:response #yada.response.Response{:representation {:media-type #yada.media-type[application/json;q=1.0], :charset #yada.charset.CharsetMap{:alias "UTF-8", :quality 1.0}}, :vary #{:media-type}}, :resource #function[backend.routes.examples.ts/fn--57734]}
;; :at [clojure.core$ex_info invoke "core.clj" 4593]}
;; {:type java.lang.IllegalArgumentException
;; :message "No implementation of method: :to-body of protocol: #'yada.body/MessageBody found for class: manifold.stream.async.CoreAsyncSource"
;; :at [clojure.core$_cache_protocol_fn invoke "core_deftype.clj" 554]}]
;; :trace
Third attempt, with a core.async channel (different error 500) :
(def get-stream
(yada (fn [ctx]
(-> (:response ctx)
(assoc :status 202)
(assoc :body (chan)))
{:representations [{:media-type "application/json"
:charset "UTF-8"}
{:media-type "application/edn"
:charset "UTF-8"}]}))
;; Error on the page :
;; 500: Unknown
;; Error on GET
;; #error {
;; :cause "No implementation of method: :to-body of protocol: #'yada.body/MessageBody found for class: clojure.core.async.impl.channels.ManyToManyChannel"
;; :via
;; [{:type clojure.lang.ExceptionInfo
;; :message "Error on GET"
;; :data {:response #yada.response.Response{:representation {:media-type #yada.media-type[application/json;q=1.0], :charset #yada.charset.CharsetMap{:alias "UTF-8", :quality 1.0}}, :vary #{:media-type}}, :resource #function[backend.routes.api.subscribe/subscribe$fn--64130]}
;; :at [clojure.core$ex_info invoke "core.clj" 4593]}
;; {:type java.lang.IllegalArgumentException
;; :message "No implementation of method: :to-body of protocol: #'yada.body/MessageBody found for class: clojure.core.async.impl.channels.ManyToManyChannel"
;; :at [clojure.core$_cache_protocol_fn invoke "core_deftype.clj" 554]}]
;; :trace
The key error is this:
"No implementation of method: :to-body of protocol:
#'yada.body/MessageBody
found for class: manifold.stream.async.CoreAsyncSource"
It reveals that the yada version you are using is trying to coerce a body type it doesn't understand. More recent versions of yada are more permissive about what you can send through to the web-server, and allow anything through that it doesn't know how to transform.

Java Interop -- Netty + Clojure

I'm trying to use netty via clojure. I'm able to startup the server, however, it fails to initialize an accepted socket. Below are the error message and code respectively. Does anyone know what is/or could be wrong? I believe the issue is with (Channels/pipeline (server-handler)) Thanks.
Error Message
#<NioServerSocketChannel [id: 0x01c888d9, /0.0.0.0:843]>
Jun 6, 2012 12:15:35 PM org.jboss.netty.channel.socket.nio.NioServerSocketPipelineSink
WARNING: Failed to initialize an accepted socket.
java.lang.IllegalArgumentException: No matching method found: pipeline
project.clj
(defproject protocol "1.0.0-SNAPSHOT"
:description "Upload Protocol Server"
:dependencies [
[org.clojure/clojure "1.2.1"]
[io.netty/netty "3.4.5.Final"]])
core.clj
(ns protocol.core
(:import (java.net InetSocketAddress)
(java.util.concurrent Executors)
(org.jboss.netty.bootstrap ServerBootstrap)
(org.jboss.netty.channel Channels ChannelPipelineFactory SimpleChannelHandler)
(org.jboss.netty.channel.socket.nio NioServerSocketChannelFactory)
(org.jboss.netty.buffer ChannelBuffers)))
(def policy
"<content>Test</content>")
(defn server-handler
"Returns netty handler."
[]
(proxy [SimpleChannelHandler] []
(messageReceived [ctx e]
(let [ch (.getChannel e)]
(.write ch policy)
(.close ch)))
(channelConnected [ctx e]
(let [ch (.getChannel e)]
(.write ch policy)
(.close ch)))
(exceptionCaught [ctx e]
(let [ex (.getCause e)]
(println "Exception" ex)
(-> e .getChannel .close)))))
(defn setup-pipeline
"Returns channel pipeline."
[]
(proxy [ChannelPipelineFactory] []
(getPipeline []
(Channels/pipeline (server-handler)))))
(defn startup
"Starts netty server."
[port]
(let [channel-factory (NioServerSocketChannelFactory. (Executors/newCachedThreadPool) (Executors/newCachedThreadPool))
bootstrap (ServerBootstrap. channel-factory)]
(.setPipelineFactory bootstrap (setup-pipeline))
(.setOption bootstrap "child.tcpNoDelay" true)
(.setOption bootstrap "child.keepAlive" true)
(.bind bootstrap (InetSocketAddress. port))))
There are three problems with your code
Java interop with vararg Channels.channel() method.
you can make a vector of channel handlers and wrap it with (into-array ChannelHandler ..)
You can not write String objects directly to a Netty Channel.
you have to write the string to a ChannelBuffer first and write that buffer or use a StringCodecHandler.
Writing to Netty channel is asynchronus, so you can not close it immediately.
you have to register a future listener and close the channel when its done.
Here is the working code.
(ns clj-netty.core
(:import (java.net InetSocketAddress)
(java.util.concurrent Executors)
(org.jboss.netty.bootstrap ServerBootstrap)
(org.jboss.netty.buffer ChannelBuffers)
(org.jboss.netty.channel Channels ChannelFutureListener ChannelHandler ChannelPipelineFactory SimpleChannelHandler)
(org.jboss.netty.channel.socket.nio NioServerSocketChannelFactory)
(org.jboss.netty.buffer ChannelBuffers)))
(def policy
(ChannelBuffers/copiedBuffer
(.getBytes "<content>Test</content>")))
(defn server-handler
"Returns netty handler."
[]
(proxy [SimpleChannelHandler] []
(messageReceived [ctx e]
(let [ch (.getChannel e)]
(.addListener
(.write ch policy)
(ChannelFutureListener/CLOSE))))
(channelConnected [ctx e]
(let [ch (.getChannel e)]
(.addListener
(.write ch policy)
(ChannelFutureListener/CLOSE))))
(exceptionCaught [ctx e]
(let [ex (.getCause e)]
(println "Exception" ex)
(-> e .getChannel .close)))))
(defn setup-pipeline
"Returns channel pipeline."
[]
(proxy [ChannelPipelineFactory] []
(getPipeline []
(let [handler (server-handler)]
(Channels/pipeline (into-array ChannelHandler [handler]))))))
(defn startup
"Starts netty server."
[port]
(let [channel-factory (NioServerSocketChannelFactory. (Executors/newCachedThreadPool) (Executors/newCachedThreadPool))
bootstrap (ServerBootstrap. channel-factory)]
(.setPipelineFactory bootstrap (setup-pipeline))
(.setOption bootstrap "child.tcpNoDelay" true)
(.setOption bootstrap "child.keepAlive" true)
(.bind bootstrap (InetSocketAddress. port))))
Have a look at Aleph (also uses Netty) which can used to build clients and servers in many different protocols with nice Clojure API.
In 2021, the easiest way to build Clojure services on top of Netty would be to adopt Donkey which provides good Clojure interop with Vert.x which uses Netty as the backend. Donkey is relatively new and doesn't yet have a lot of documentation so check out this blog which covers a relevant open source project (source available) in terms of architecture, design, coding, and testing under load.