Function from golang c-library simply stops in C++ - c++

I am trying to setup a little server-client application.
For this purpose I created a network server and a network client in golang. I created a library from the client and I am calling the code in my C++ program, but when the program reaches the function call for the message function the program stops and never returns from the function.
I tried to send messages from golang to golang, which works fine.
Here is the go source code for the client:
package main
import (
"net"
"strconv"
"strings"
"fmt"
"C"
)
const (
StopCharacter = "\r\n\r\n"
)
//export GoMessage
func GoMessage(ip string, port int, message string) string {
address := strings.Join([]string{ip, strconv.Itoa(port)}, ":")
connection, err := net.Dial("tcp", address)
if err != nil {
fmt.Println(err)
}
defer connection.Close()
connection.Write([]byte(message))
connection.Write([]byte(StopCharacter))
buffer := make([]byte, 1024)
n, _ := connection.Read(buffer)
data := string(buffer[:n])
return data
}
func main() {
/*arg := os.Args[1]
var (
ip = "127.0.0.1"
port = 3333
)
buffer := SendMessage(ip, port, arg)
fmt.Println(buffer)
ToFile(buffer)*/
}
Here is the server code:
package main
import (
"bufio"
"io"
"fmt"
"net"
"strconv"
"strings"
"os"
)
const (
StopCharacter = "\r\n\r\n"
)
func ToFile(message string) {
f, err := os.Create("gout.txt")
defer f.Close()
if(err != nil) {
fmt.Println(err)
}
f.WriteString(message)
}
func SocketServer(port int) {
listen, err := net.Listen("tcp4", ":"+strconv.Itoa(port))
defer listen.Close()
if err != nil {
fmt.Println(err)
}
for {
connection, err := listen.Accept()
if err != nil {
fmt.Println(err)
continue
}
go handler(connection)
}
}
func handler(connection net.Conn) {
defer connection.Close()
var (
buffer = make([]byte, 1024)
r = bufio.NewReader(connection)
w = bufio.NewWriter(connection)
)
var data string = "";
ILOOP:
for {
n, err := r.Read(buffer)
data = string(buffer[:n])
switch err {
case io.EOF:
break ILOOP
case nil:
if isTransportOver(data) {
break ILOOP
}
default:
return
}
}
fmt.Println(data)
ToFile(data)
w.Write([]byte("Test123"))
w.Flush()
}
func isTransportOver(data string) (over bool) {
over = strings.HasSuffix(data, "\r\n\r\n")
return
}
func main() {
port := 3333
SocketServer(port)
}
and this is my C++ code:
#include <sstream>
#include <iostream>
#include <conio.h>
#include "goclient.h"
extern GoString GoMessage(GoString ip, GoInt port, GoString message);
std::string message(GoString message)
{
GoString ip;
ip.p = "127.0.0.1";
ip.n = 10;
GoString answer = GoMessage(ip, 3333, message);
return answer.p;
}
int main()
{
GoString msg;
msg.p = "hi";
msg.n = 2;
std::cout << message(msg);
_getch();
return 0;
}
I am doing this just for the sake of sience... I am aware that there are better solutions for this (like thrift).
I am completely out of ideas what to do to solve the problem.
I am using msvc btw.
Update:
First of all I tried to compile & run the program with minGW, which brought up a NUL string error:
Sorry can't copy from mingwshell)
Afterwards I tried to return from the GoMessage function direct after calling:
//export GoMessage
func GoMessage(ip string, port int, message string) string {
return "test"
address := strings.Join([]string{ip, strconv.Itoa(port)}, ":")
connection, err := net.Dial("tcp", address)
if err != nil {
fmt.Println(err)
}
defer connection.Close()
connection.Write([]byte(message))
connection.Write([]byte(StopCharacter))
buffer := make([]byte, 1024)
n, _ := connection.Read(buffer)
data := string(buffer[:n])
return data
}
Which lead to the same result on msvc and but a different output on minGW:
(Sorry can't copy from mingwshell)

Related

RDS access via IAM role from a Golang microservice

This bounty has ended. Answers to this question are eligible for a +50 reputation bounty. Bounty grace period ends in 21 hours.
Abhisek Roy wants to draw more attention to this question:
Looking for a workaround to the current issue.
I was implementing RDS IAM authentication from my Golang service and came across an issue. I am using the init function to call create a function where I have created a sync for every 10 mins where every 10 mins the token should get refreshed, it is creating the new token but it's not passing the same to the main function. When the existing thread that was initiated between DB and the service gets killed it is unable to re-authenticate the connection for the second time. Some observations that I have noticed were-
When initially deploying the application it is creating the connection to RDS successfully but after some time as soon as the initial thread between service and DB gets killed it's unable to authenticate the reason which I checked was- the address where the token is getting stored. While making an API call the service picks the token from a static address where the initial token gets stored at the time of service deployment. However, while the token gets refreshed every 10 mins its getting stored on dynamic addresses from where the service is unable to pick up the token.
Here is the go file where actually I am calling and creating the DB function-
package db
import (
"fmt"
"log"
"os"
"time"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/rds/rdsutils"
"github.com/go-co-op/gocron"
"github.com/golang-migrate/migrate/v4"
"github.com/golang-migrate/migrate/v4/database/postgres"
_ "github.com/golang-migrate/migrate/v4/source/file"
"github.com/jinzhu/gorm"
_ "github.com/lib/pq"
)
const DatabaseVersion = 1
const Success = "Database connection successful"
const database = "postgres"
// Service ...
type Service interface {
InitDatabaseConnection(db **gorm.DB)
CreateDatabaseConnection(db **gorm.DB)
Migrate(db *gorm.DB) error
}
type service struct {
}
// NewService ...
func NewService() Service {
return &service{}
}
func (s *service) InitDatabaseConnection(db **gorm.DB) {
log.Print("Creating Db Connection")
s.CreateDatabaseConnection(db)
//todo: add if; if env for iam run the below code
// todo: change 1 min to 10 mins
startTime := time.Now().Local().Add(time.Minute + time.Duration(1))
sch := gocron.NewScheduler(time.UTC)
sch.Every(600).Seconds().StartAt(startTime).Do(func() {
log.Printf("refreshing rds creds")
s.CreateDatabaseConnection(db)
})
sch.StartAsync()
}
func (s *service) CreateDatabaseConnection(db **gorm.DB) {
configuration, _ := common.New()
var dbURI = ConnectDataBase(configuration.UserName, configuration.Password,
configuration.Port, configuration.DbName, configuration.Host, configuration.SearchPath)
var err error
*db, err = gorm.Open(database, dbURI)
if err != nil {
panic(err)
}
print("\n \n", dbURI)
print("\n \n in server.go \n \n ", &db)
if err != nil {
log.Println(err.Error())
os.Exit(3)
} else {
log.Println(Success)
}
(*db).DB().SetMaxOpenConns(6)
(*db).DB().SetConnMaxLifetime(600 * time.Second)
(*db).DB().SetMaxIdleConns(2)
(*db).DB().SetConnMaxIdleTime(100 * time.Second)
}
func ConnectDataBase(username string, password string, port int, dbName string, host string, searchPath string) string {
var dbURI string
if searchPath == "" {
searchPath = "public"
}
dbEndpoint := fmt.Sprintf("%s:%d", host, port)
sess := session.Must(session.NewSession())
creds := sess.Config.Credentials
authToken, err := rdsutils.BuildAuthToken(dbEndpoint, "us-east-1", username, creds)
print("\n \n", authToken)
if err != nil {
panic(err)
}
dbURI = fmt.Sprintf("host=%s port=%d user=%s dbname=%s sslmode=require password=%s search_path=%s",
host, port, username, dbName, authToken, searchPath) //Build connection string
// if password != "" && username != "" {
// dbURI = fmt.Sprintf("host=%s port=%d user=%s dbname=%s sslmode=require password=%s search_path=%s",
// host, port, username, dbName, authToken, searchPath) //Build connection string
// } else {
// dbURI = fmt.Sprintf("host=%s port=%d dbname=%s sslmode=disable ",
// host, port, dbName) //Build connection string
// }
return dbURI
}
func (s *service) Migrate(db *gorm.DB) error {
migrationSourceURL := "file://../resources/db/migrations/"
database := db.DB()
row := db.Table("schema_migrations").Limit(1).Row()
var version int8
var dirty bool
err := row.Scan(&version, &dirty)
if err == nil {
log.Printf("database is currently on version : %v \n", version)
}
log.Println("Migrating database to version ", DatabaseVersion)
driver, err := postgres.WithInstance(database, &postgres.Config{})
if err != nil {
return err
}
m, err := migrate.NewWithDatabaseInstance(migrationSourceURL, "postgres", driver)
if err != nil {
return err
}
m.Log = LogService{}
err = m.Migrate(uint(DatabaseVersion))
if err != nil {
log.Println(err.Error())
return err
}
return nil
}
The common property above is from this piece of code-
package common
import (
"fmt"
"log"
"os"
"github.com/spf13/viper"
)
type ApiResponse struct {
Code int `json:"code"`
Message string `json:"message"`
}
type Constants struct {
Port int `json:"port"`
Host string `json:"host"`
DbName string `json:"dbName"`
UserName string `json:"userName"`
Password string `json:"password"`
AuthUserName string `json:"authUserName"`
AuthPassword string `json:"authPassword"`
SearchPath string `json:"searchPath"`
AwsRegion string `json:"awsRegion"`
}
func New() (*Constants, error) {
fmt.Println("reached constants")
config := Constants{}
constants, err := initViper()
config = constants
if err != nil {
return &config, err
}
return &config, nil
}
func initViper() (Constants, error) {
// temporary, will be removed later when auto deploy is available
env := os.Getenv("CS_ENV")
var configName string
if env == "develop" {
configName = "config-develop"
} else {
configName = "config-local"
}
viper.SetConfigName(configName) // Constants fileName without the .TOML or .YAML extension
viper.AddConfigPath("../resources/config") // Search the root directory for the configuration file
err := viper.ReadInConfig() // Find and read the config file
if err != nil {
log.Println(err.Error()) // Handle errors reading the config file
return Constants{}, err
}
if err = viper.ReadInConfig(); err != nil {
log.Panicf("Error reading config file, %s", err)
}
var constants Constants
err = viper.Unmarshal(&constants)
return constants, err
}
Also attaching the main function go file where I am calling the DB function for connecting the service to DB-
package main
import (
"bufio"
"encoding/base64"
"encoding/json"
"fmt"
"log"
"net/http"
"os"
"strings"
"time"
gokitlog "github.com/go-kit/kit/log"
"github.com/go-kit/kit/log/level"
"github.com/gorilla/mux"
"github.com/jinzhu/gorm"
_ "github.com/lib/pq"
)
func runServer(config *common.Constants) {
var logger gokitlog.Logger
logger = gokitlog.NewLogfmtLogger(gokitlog.NewSyncWriter(os.Stderr))
logger = gokitlog.With(logger, "ts", gokitlog.DefaultTimestampUTC)
dbService := db2.NewService()
var db *gorm.DB
dbService.InitDatabaseConnection(&db)
err := dbService.Migrate(db)
if err != nil {
fmt.Println("Migrations scripts did not run :", err.Error())
if err.Error() != "no change" {
fmt.Println("failed to run migrations hence exiting", err.Error())
os.Exit(1)
}
}
level.Info(logger).Log("Starting Darwin ==========>")
repo, err := allocation.NewUserAllocationRepository(db, logger)
if err != nil {
level.Error(logger).Log("exit", err)
os.Exit(3)
}
userAllocationService := allocation.NewUserAllocationService(repo, logger)
endpoints := allocation.MakeEndpoints(userAllocationService)
router := mux.NewRouter()
sh := http.StripPrefix("/swaggerui/", http.FileServer(http.Dir("../resources/swaggerui/")))
router.PathPrefix("/swaggerui/").Handler(sh)
httpLogger := gokitlog.With(logger, "component", "http")
subRouter := router.PathPrefix("/api/v1").Subrouter()
subRouter.NotFoundHandler = http.HandlerFunc(notFound)
subRouter.Use(checkBasicAuth(config))
subRouter.HandleFunc("/healthCheck", healthCheck).Methods("GET")
allocation.MakeHandler(subRouter, httpLogger, endpoints)
http.Handle("/", subRouter)
log.Print("Running server on port 8080")
f, err := os.OpenFile("../performance.log", os.O_RDWR|os.O_CREATE|os.O_APPEND, 0666)
if err != nil {
log.Printf("error opening file: %v", err)
log.Fatal(http.ListenAndServe(":8080", router))
} else {
by := bufio.NewWriter(f)
defer f.Close()
log.Fatal(http.ListenAndServe(":8080", Logger(by, router)))
}
}
func healthCheck(w http.ResponseWriter, r *http.Request) {
resp := map[string]string{
"Status": "Success",
}
w.Header().Set("Content-Type", "application/json; charset=utf-8")
w.WriteHeader(http.StatusOK)
json.NewEncoder(w).Encode(resp)
}
func notFound(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json;")
w.WriteHeader(http.StatusNotFound)
json.NewEncoder(w).Encode(map[string]string{"error": "api not found"})
}
func authHandler(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json;")
w.WriteHeader(http.StatusUnauthorized)
json.NewEncoder(w).Encode(map[string]string{"error": "Not authorized"})
}
func checkBasicAuth(config *common.Constants) func(http.Handler) http.Handler {
return func(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.RequestURI() != "/api/v1/healthCheck" {
w.Header().Set("WWW-Authenticate", `Basic realm="Restricted"`)
w.Header().Set("Content-Type", "application/json;")
s := strings.SplitN(r.Header.Get("Authorization"), " ", 2)
if len(s) != 2 {
authHandler(w, r)
return
}
b, err := base64.StdEncoding.DecodeString(s[1])
if err != nil {
authHandler(w, r)
return
}
pair := strings.SplitN(string(b), ":", 2)
if len(pair) != 2 {
authHandler(w, r)
return
}
if pair[0] != config.AuthUserName || pair[1] != config.AuthPassword {
authHandler(w, r)
return
}
}
next.ServeHTTP(w, r)
})
}
}
func Logger(out *bufio.Writer, h http.Handler) http.Handler {
logger := log.New(out, "", 0)
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
o := &responseObserver{ResponseWriter: w, status: http.StatusOK}
start := time.Now()
h.ServeHTTP(o, r)
endTime := time.Now()
if !strings.Contains(r.URL.String(), "/health") && strings.Contains(r.URL.String(), "/api") {
logger.Printf("Method: %s Path: %s Status: %d ExecutionTime: %d ms", r.Method, r.URL, o.status, endTime.Sub(start).Milliseconds())
out.Flush()
}
})
}
type responseObserver struct {
http.ResponseWriter
status int
}
func (r *responseObserver) WriteHeader(status int) {
r.status = status
r.ResponseWriter.WriteHeader(status)
}

Unit-testing grpc functions in golang

I have created a function that utilizes the grpc package in golang. I don't know if it is relevant but the purpose is the communication with a GoBGP router over grpc. An example is the following function which prints all the peers (neighbors) of the router:
func (gc *Grpc) Peers(conn *grpc.ClientConn) error {
defer conn.Close()
c := pb.NewGobgpApiClient(conn)
ctx, cancel := context.WithTimeout(context.Background(), time.Second*10)
defer cancel()
p := pb.ListPeerRequest{}
peer, err := c.ListPeer(ctx, &p)
if err != nil {
return err
}
for {
res, err := peer.Recv()
if err != nil {
return err
}
fmt.Println(res)
}
return nil
}
Now, I want to create unit tests for the function. To do so, I used google.golang.org/grpc/test/bufconn package, and initialized the following:
type server struct {
pb.UnimplementedGobgpApiServer
}
func (s *server) ListDefinedSet(in *pb.ListDefinedSetRequest, ls pb.GobgpApi_ListDefinedSetServer) error {
return nil
}
var lis *bufconn.Listener
const bufSize = 1024 * 1024
func init() {
lis = bufconn.Listen(bufSize)
s := grpc.NewServer()
pb.RegisterGobgpApiServer(s, &server{})
go func() {
if err := s.Serve(lis); err != nil {
fmt.Println("Server failed!")
}
}()
}
func bufDialer(context.Context, string) (net.Conn, error) {
return lis.Dial()
}
This way, I can run a unit-test creating a connection as follows:
ctx := context.Background()
conn, _ := grpc.DialContext(ctx, "bufnet", grpc.WithContextDialer(bufDialer), grpc.WithInsecure())
Peers(conn)
However, the problem is that the stream seems to be always empty and thus the peer.Recv()
always returns EOF. Is there any way to populate the stream with dummy data? If you have experience, is my methodology correct?

Using interface for testing like dependency injection

I use the following code which works ok.
This is working example
https://play.golang.org/p/wjvJtDNvJAQ
package main
import (
"encoding/json"
"fmt"
"io/ioutil"
"net/http"
)
type requester interface {
HTTPRequest(c string, i string, mtd string, url string) (p []byte, e error)
}
type impl struct {
client *http.Client
}
// ----This is the function which I need to mock
func (s *ServiceInfo) wrapperFN() {
// Function 1 - get the values
v1, v2 := s.json.parseJson()
// call to http function
s.req.HTTPRequest(v1, v2, "POST", "http://www.mocky.io/v2/5c20eccc2e00005c001e0c84")
}
func (i impl) HTTPRequest(c string, ci string, mtd string, url string) (p []byte, e error) {
req, err := http.NewRequest(mtd, url, nil)
if err != nil {
return nil, err
}
req.SetBasicAuth(c, ci)
res, err := i.client.Do(req)
if err != nil {
return nil, err
}
token, err := ioutil.ReadAll(res.Body)
if err != nil {
return nil, err
}
defer res.Body.Close()
fmt.Println("success")
return token, nil
}
type parser interface {
parseJson() (string, string)
}
type jsonP struct {
data string
}
func (s jsonP) parseJson() (string, string) {
var result map[string]interface{}
json.Unmarshal([]byte(s.data), &result)
b := result["person"].(map[string]interface{})
for key, value := range b {
return key, value.(string)
}
return "", ""
}
type ServiceInfo struct {
req requester
json parser
}
// When in production pass in concrete implementations.
func NewServiceInfo(http requester, json parser) *ServiceInfo {
return &ServiceInfo{
req: http,
json: json,
}
}
func main() {
httpClient := http.Client{}
js := `{"person":{"p1":"username","p2":"password"},"customers":"10"}`
j := jsonP{data: js}
s := NewServiceInfo(impl{client: &httpClient}, j)
s.wrapperFN()
}
Now i want to test it wrapperFN , what I try I've changed the code to use interface , which works.
This is just example to give a point ( the real code much more complicated)
The problem that I dont understand how to mock function inside wrapperFN like parseJson() , in the real world warpperFN contains several function which I need to mock ,because just calling them in the test will provide error.
How it's best to mock function like parseJson() & HTTPRequest? and assume that inside wrapperFN there is additional functions which is not related...
I need to know if this is the best practice for testing function.
This is the test (which im not sure how to make it right)
package main
import (
"net/http"
"net/http/httptest"
"testing"
)
func TestServiceInfo_wrapperFN(t *testing.T) {
tests := []struct {
name string
s *ServiceInfo
}{
{
name: "wrapper test",
s: &ServiceInfo{},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
var testHandler http.Handler
srv := httptest.NewServer(testHandler)
defer srv.Close()
iReq := &impl{
client: srv.Client(),
}
v := &ServiceInfo{http: *iReq}
v.wrapperFN()
})
}
}

Testing with Gomock returns error: Expected call has already been called the max number of times

I am using Gomock https://godoc.org/github.com/golang/mock and mockgen
The Source code for this test is:
package sqs
import (
"fmt"
"log"
"os"
"runtime"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/service/sqs"
"github.com/aws/aws-sdk-go/service/sqs/sqsiface"
)
var sess *session.Session
var svc *sqs.SQS
var queueURL string
func init() {
// Setting the runtime to run with max CPUs available
runtime.GOMAXPROCS(runtime.NumCPU())
sess = session.Must(session.NewSessionWithOptions(session.Options{
SharedConfigState: session.SharedConfigEnable,
}))
svc = sqs.New(sess)
queueURL = os.Getenv("QUEUE_URL")
}
type Poller interface {
Poll(chan bool)
}
// NewPoller is a factory to create a Poller object
func NewPoller(msgr Messenger) Poller {
p := &poller{
m: msgr,
}
return p
}
type poller struct {
m Messenger
}
func (p *poller) Poll(done chan bool) {
sqsMsgCh := make(chan *sqs.Message, 100)
for {
messages, err := p.m.GetMessage()
if err != nil {
log.Printf("error when getting message")
if len(messages) == 0 {
// Stop the system
log.Printf("I am here")
done <- true
}
}
for _, msg := range messages {
sqsMsgCh <- msg
}
}
}
type Messenger interface {
GetMessage() ([]*sqs.Message, error)
}
func NewMessenger() Messenger {
return &messenger{
s: svc,
}
}
type messenger struct {
s sqsiface.SQSAPI
}
func (m *messenger) GetMessage() ([]*sqs.Message, error) {
result, err := m.s.ReceiveMessage(&sqs.ReceiveMessageInput{
AttributeNames: []*string{
aws.String(sqs.MessageSystemAttributeNameSentTimestamp),
},
MessageAttributeNames: []*string{
aws.String(sqs.QueueAttributeNameAll),
},
QueueUrl: aws.String(queueURL),
MaxNumberOfMessages: aws.Int64(10),
VisibilityTimeout: aws.Int64(36000), // 10 hours
WaitTimeSeconds: aws.Int64(0),
})
if err != nil {
fmt.Println("Error", err)
return nil, err
}
msgs := result.Messages
if len(msgs) == 0 {
fmt.Println("Received no messages")
return msgs, err
}
return msgs, nil
}
The test case for this Source file is here:
package sqs
import (
"errors"
"testing"
"path_to_the_mocks_package/mocks"
"github.com/golang/mock/gomock"
"github.com/aws/aws-sdk-go/service/sqs"
)
func TestPollWhenNoMessageOnQueue(t *testing.T) {
mockCtrl := gomock.NewController(t)
defer mockCtrl.Finish()
msgr := mocks.NewMockMessenger(mockCtrl)
mq := make([]*sqs.Message, 1)
err := errors.New("Mock Error")
// msgr.EXPECT().GetMessage().Return(mq, err) //.Times(1)
// msgr.GetMessage().Return(mq, err) //.Times(1)
msgr.EXPECT().GetMessage().Return(mq, err)
p := NewPoller(msgr)
done := make(chan bool)
go p.Poll(done)
<-done
t.Logf("Successfully done: %v", done)
}
When I run the tests I am getting the following error:
sqs\controller.go:150: Unexpected call to
*mocks.MockMessenger.GetMessage([]) at path_to_mocks_package/mocks/mock_messenger.go:38 because: Expected
call at path_to_sqs_package/sqs/sqs_test.go:35 has already been called
the max number of times. FAIL
If I write my own mock as follows the test case executes successfully:
type mockMessenger struct {
mock.Mock
}
func (m *mockMessenger) GetMessage() ([]*sqs.Message, error) {
msgs := make([]*sqs.Message, 0)
err := errors.New("Error")
return msgs, err
}
You are implicitly telling gomock that you only expect a single call.
msgr.EXPECT().GetMessage().Return(mq, err)
Adding a number of Times to the mock, allows you to return those values more than once.
msgr.EXPECT().GetMessage().Return(mq, err).AnyTimes()
For more details please read the gomock's AnyTimes documentation.

how can I reduce cpu usage in a golang tcp server?

I try to implement a golang tcp server, and I found the concurrency is satisfied for me, but the CPU usage is too high(concurrency is 15W+/s, but the CPU usage is about 800% in a 24 cores linux machine). At the same time, a C++ tcp server is only about 200% usage with a similar concurrency(with libevent).
The following code is the demo of golang:
func main() {
listen, err := net.Listen("tcp", "0.0.0.0:17379")
if err != nil {
fmt.Errorf(err.Error())
}
go acceptClient(listen)
var channel2 = make(chan bool)
<-channel2
}
func acceptClient(listen net.Listener) {
for {
sock, err := listen.Accept()
if err != nil {
fmt.Errorf(err.Error())
}
tcp := sock.(*net.TCPConn)
tcp.SetNoDelay(true)
var channel = make(chan bool, 10)
go read(channel, sock.(*net.TCPConn))
go write(channel, sock.(*net.TCPConn))
}
}
func read(channel chan bool, sock *net.TCPConn) {
count := 0
for {
var buf = make([]byte, 1024)
n, err := sock.Read(buf)
if err != nil {
close(channel)
sock.CloseRead()
return
}
count += n
x := count / 58
count = count % 58
for i := 0; i < x; i++ {
channel <- true
}
}
}
func write(channel chan bool, sock *net.TCPConn) {
buf := []byte("+OK\r\n")
defer func() {
sock.CloseWrite()
recover()
}()
for {
_, ok := <-channel
if !ok {
return
}
_, writeError := sock.Write(buf)
if writeError != nil {
return
}
}
}
And I test this tcp server by the redis-benchmark with multi-clients:
redis-benchmark -h 10.100.45.2 -p 17379 -n 1000 -q script load "redis.call('set','aaa','aaa')"
I also analyzed my golang code by the pprof, it is said CPU cost a lot of time on syscall:
enter image description here
I don't think parallelise the read and write with channel will provide you better performance in this case. You should try to do less memory allocation and less syscall (The write function may do a lot of syscalls)
Can you try this version?
package main
import (
"bytes"
"fmt"
"net"
)
func main() {
listen, err := net.Listen("tcp", "0.0.0.0:17379")
if err != nil {
fmt.Errorf(err.Error())
}
acceptClient(listen)
}
func acceptClient(listen net.Listener) {
for {
sock, err := listen.Accept()
if err != nil {
fmt.Errorf(err.Error())
}
tcp := sock.(*net.TCPConn)
tcp.SetNoDelay(true)
go handleConn(tcp) // less go routine creation but no concurrent read/write on the same conn
}
}
var respPattern = []byte("+OK\r\n")
// just one goroutine per conn
func handleConn(sock *net.TCPConn) {
count := 0
buf := make([]byte, 4098) // Do not create a new buffer each time & increase the buff size
defer sock.Close()
for {
n, err := sock.Read(buf)
if err != nil {
return
}
count += n
x := count / 58
count = count % 58
resp := bytes.Repeat(respPattern, x) // can be optimize
_, writeError := sock.Write(resp) // do less syscall
if writeError != nil {
return
}
}
}