I am trying to seek till the end of a file using fseek, seeking in steps of 2560 * 5 bytes at a time. No idea why I am getting a segmentation fault
I am running this code on TDA2x board. Is here a problem in the particular environment which I am unaware of?
Starting value of uNumFrames is 1000
This code is basically trying to read the data from an externally connected SSD drive, and is reading the file byte by byte, skipping certain containers in the file using fseek. The problem I am facing is that while trying to read the files like this, the code is crashing with the following details
while(uNumFrames > 500)
{
logger::addLog(logger::LOGGER_INFO,"Inside while loop, uNumFrames = %d", uNumFrames);
/*Skip packet Header*/
ui_skip_count = fseek(fp, HeaderSize * 5, SEEK_CUR);
fsize = ftell(fp);
logger::addLog(logger::LOGGER_INFO,"fsize = %ld\n", fsize);
if (ferror(fp))
{
logger::addLog(logger::LOGGER_INFO,"fseek Error");
fclose (fp);
break;
}
if(0 != ui_skip_count)
{
logger::addLog(logger::LOGGER_INFO,"fseek for fp failed");
}
else
{
logger::addLog(logger::LOGGER_INFO,"Inside else for read and scale");
}
uNumFrames--;
}
Sample output looks something like this:
[HOST ] [INFO] 86.234680 s: Inside while loop, uNumFrames = 978
[HOST ] [INFO] 86.234680 s: fsize = 3368961
[HOST ] [INFO] 86.234710 s: Inside else for read and scale
[HOST ] [INFO] 86.234710 s: Inside while loop, uNumFrames = 977
[HOST ] [INFO] 86.234710 s: fsize = 3381761
[HOST ] [INFO] 86.234710 s: Inside else for read and scale
[HOST ] [INFO] 86.234710 s: Inside while loop, uNumFrames = 976
[HOST ] [INFO] 86.234741 s: fsize = 3394561
[HOST ] [INFO] 86.234741 s: Inside else for read and scale
[HOST ] [INFO] 86.234741 s: Inside while loop, uNumFrames = 975
[HOST ] [INFO] 86.234741 s: fsize = 3407361
[HOST ] [INFO] 86.234741 s: Inside else for read and scale
[HOST ] [INFO] 86.234741 s: Inside while loop, uNumFrames = 974
[HOST ] [INFO] 86.234771 s: fsize = 3420161
[HOST ] [INFO] 86.234771 s: Inside else for read and scale
[HOST ] [INFO] 86.234771 s: Inside while loop, uNumFrames = 973
[HOST ] [INFO] 86.234771 s: fsize = 3432961
[HOST ] [INFO] 86.234771 s: Inside else for read and scale
[HOST ] [INFO] 86.234771 s: Inside while loop, uNumFrames = 972
[HOST ] [INFO] 86.234802 s: fsize = 3445761
[HOST ] [INFO] 86.234802 s: Inside else for read and scale
[HOST ] [INFO] 86.234802 s: Inside while loop, uNumFrames = 971
[HOST ] [INFO] 86.234802 s: fsize = 3458561
[HOST ] [INFO] 86.234802 s: Inside else for read and scale
****** Segmentation fault caught ....
Faulty address is 0xa6499020, called from 0x77ddb
Totally Obtained 0 stack frames. signal number =11
Signal number = 11, Signal errno = 0
SI code = 1 (Address not mapped to object)
Fault addr = 0xa6499020
[bt] Execution path:
Related
i'm trying to run a test command in sbt-shell of intellij ide for this -> [https://github.com/theiterators/akka-http-microservice#akka-http-microservice-example] project then it shows Akka version conflict. how to resolve it?
this is build.sbt
enablePlugins(JavaAppPackaging , GatlingPlugin)
name := "akka-http-microservice"
organization := "com.theiterators"
version := "1.0"
scalaVersion := "2.13.5"
scalacOptions := Seq("-unchecked", "-deprecation", "-encoding", "utf8",
"-target:jvm-1.8",
"-feature",
"-language:implicitConversions",
"-language:postfixOps")
libraryDependencies ++= {
val akkaHttpV = "10.2.4"
val akkaV = "2.6.14"
val scalaTestV = "3.2.8"
val circeV = "0.13.0"
val akkaHttpCirceV = "1.36.0"
val gatlingVersion = "3.5.1"
Seq(
"com.typesafe.akka" %% "akka-actor" % akkaV,
"com.typesafe.akka" %% "akka-stream" % akkaV,
"com.typesafe.akka" %% "akka-http" % akkaHttpV,
"io.circe" %% "circe-core" % circeV,
"io.circe" %% "circe-generic" % circeV,
"de.heikoseeberger" %% "akka-http-circe" % akkaHttpCirceV,
"com.typesafe.akka" %% "akka-testkit" % akkaV,
"com.typesafe.akka" %% "akka-http-testkit" % akkaHttpV % "test",
"org.scalatest" %% "scalatest" % scalaTestV % "test",
"io.gatling.highcharts" % "gatling-charts-highcharts" % gatlingVersion % "test,it",
"io.gatling" % "gatling-test-framework" % gatlingVersion % "test,it"
)
}
Revolver.settings
this is plugin.sbt
addSbtPlugin("io.spray" % "sbt-revolver" % "0.9.1")
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.15.0")
addSbtPlugin("com.typesafe.sbt" % "sbt-native-packager" % "1.8.1")
addSbtPlugin("io.gatling" % "gatling-sbt" % "3.2.2")
addSbtPlugin("io.gatling" % "gatling-sbt" % "MANUALLY_REPLACE_WITH_LATEST_VERSION")
this is service specification
import akka.event.NoLogging
import akka.http.scaladsl.model.ContentTypes._
import akka.http.scaladsl.model.{HttpRequest, HttpResponse}
import akka.http.scaladsl.model.StatusCodes._
import akka.http.scaladsl.testkit.ScalatestRouteTest
import akka.stream.scaladsl.Flow
import org.scalatest.flatspec.AsyncFlatSpec
import org.scalatest.matchers.should.Matchers
class ServiceSpec extends AsyncFlatSpec with Matchers with ScalatestRouteTest with Service with Protocols {
override def testConfigSource = "akka.loglevel = WARNING"
override def config = testConfig
override val logger = NoLogging
val ip1Info = IpInfo("8.8.8.8", Option("United States"), Option("Mountain View"), Option(37.386), Option(-122.0838))
val ip2Info = IpInfo("8.8.4.4", Option("United States"), None, Option(38.0), Option(-97.0))
val ipPairSummary = IpPairSummary(ip1Info, ip2Info)
override lazy val ipApiConnectionFlow = Flow[HttpRequest].map { request =>
if (request.uri.toString().endsWith(ip1Info.query))
HttpResponse(status = OK, entity = marshal(ip1Info))
else if(request.uri.toString().endsWith(ip2Info.query))
HttpResponse(status = OK, entity = marshal(ip2Info))
else
HttpResponse(status = BadRequest, entity = marshal("Bad ip format"))
}
"Service" should "respond to single IP query" in {
Get(s"/ip/${ip1Info.query}") ~> routes ~> check {
status shouldBe OK
contentType shouldBe `application/json`
responseAs[IpInfo] shouldBe ip1Info
}
Get(s"/ip/${ip2Info.query}") ~> routes ~> check {
status shouldBe OK
contentType shouldBe `application/json`
responseAs[IpInfo] shouldBe ip2Info
}
}
it should "respond to IP pair query" in {
Post(s"/ip", IpPairSummaryRequest(ip1Info.query, ip2Info.query)) ~> routes ~> check {
status shouldBe OK
contentType shouldBe `application/json`
responseAs[IpPairSummary] shouldBe ipPairSummary
}
}
it should "respond with bad request on incorrect IP format" in {
Get("/ip/asdfg") ~> routes ~> check {
status shouldBe BadRequest
responseAs[String].length should be > 0
}
Post(s"/ip", IpPairSummaryRequest(ip1Info.query, "asdfg")) ~> routes ~> check {
status shouldBe BadRequest
responseAs[String].length should be > 0
}
Post(s"/ip", IpPairSummaryRequest("asdfg", ip1Info.query)) ~> routes ~> check {
status shouldBe BadRequest
responseAs[String].length should be > 0
}
}
}
Error given below--
[info] ServiceSpec *** ABORTED ***
[info] java.lang.IllegalStateException: You are using version 2.6.14 of Akka, but it appears you (perhaps indirectly) also depend on older versions of related artifacts. You can solve this by adding an explicit dependency on version 2.6.14 of the [akka-slf4j] artifacts to your project. Here's a complete collection of detected artifacts: (2.6.11, [akka-slf4j]), (2.6.14, [akka-actor, akka-protobuf-v3, akka-stream, akka-testkit]). See also: https://doc.akka.io/docs/akka/current/common/binary-compatibility-rules.html#mixed-versioning-is-not-allowed
[info] at akka.util.ManifestInfo.checkSameVersion(ManifestInfo.scala:184)
[info] at akka.util.ManifestInfo.checkSameVersion(ManifestInfo.scala:162)
[info] at akka.actor.ActorSystemImpl.liftedTree2$1(ActorSystem.scala:1033)
[info] at akka.actor.ActorSystemImpl._start$lzycompute(ActorSystem.scala:1022)
[info] at akka.actor.ActorSystemImpl._start(ActorSystem.scala:1022)
[info] at akka.actor.ActorSystemImpl.start(ActorSystem.scala:1045)
[info] at akka.actor.ActorSystem$.apply(ActorSystem.scala:272)
[info] at akka.actor.ActorSystem$.apply(ActorSystem.scala:316)
[info] at akka.actor.ActorSystem$.apply(ActorSystem.scala:290)
[info] at akka.http.scaladsl.testkit.RouteTest.createActorSystem(RouteTest.scala:36)
[info] ...
[info] Run completed in 1 second, 568 milliseconds.
[info] Total number of tests run: 0
[info] Suites: completed 0, aborted 1
[info] Tests: succeeded 0, failed 0, canceled 0, ignored 0, pending 0
[info] *** 1 SUITE ABORTED ***
[error] Error during tests:
[error] ServiceSpec
[error] (Test / test) sbt.TestsFailedException: Tests unsuccessful
[error] Total time: 8 s, completed 22 Apr. 2021, 3:14:59 pm
Adding
"com.typesafe.akka" %% "akka-slf4j" % akkaV
to the Seq in your libraryDependencies block should resolve this. In general in Akka it's best to not rely on transitive dependencies.
I am facing a performance issue in Akka remoting. I have 2 actors Actor1 and Actor2. The message sending between the actor is synchronous ask request from Actor1 to Actor2 and the response back from Actor2 to Actor1. Below is the sample code snippets and config of my Actor:
Actor1.java:
object Actor1 extends App {
val conf = ConfigFactory.load()
val system = ActorSystem("testSystem1", conf.getConfig("remote1"))
val actor = system.actorOf(Props[Actor1].withDispatcher("my-dispatcher"), "actor1")
implicit val timeOut: Timeout = Timeout(10 seconds)
class Actor1 extends Actor {
var value = 0
var actorRef: ActorRef = null
override def preStart(): Unit = {
println(self.path)
}
override def receive: Receive = {
case "register" =>
actorRef = sender()
println("Registering the actor")
val time = System.currentTimeMillis()
(1 to 300000).foreach(value => {
if (value % 10000 == 0) {
println("message count -- " + value + " --- time taken - " + (System.currentTimeMillis() - time))
}
Await.result(actorRef ? value, 10 seconds)
})
val totalTime = System.currentTimeMillis() - time
println("Total Time - " + totalTime)
}
}
}
Actor2.java:
object Actor2 extends App {
val conf = ConfigFactory.load()
val system = ActorSystem("testSystem1", conf.getConfig("remote2"))
val actor = system.actorOf(Props[Actor2].withDispatcher("my-dispatcher"), "actor2")
implicit val timeOut: Timeout = Timeout(10 seconds)
actor ! "send"
class Actor2 extends Actor {
var value = 0
var actorSelection: ActorSelection = context.actorSelection("akka://testSystem1#127.0.0.1:6061/user/actor1")
override def receive: Receive = {
case "send" =>
actorSelection ! "register"
case int: Int => {
sender() ! 1
}
}
}
}
application.conf:
remote1 {
my-dispatcher {
executor = "thread-pool-executor"
type = PinnedDispatcher
}
akka {
actor {
provider = remote
}
remote {
artery {
transport = tcp # See Selecting a transport below
canonical.hostname = "127.0.0.1"
canonical.port = 6061
}
}
}
}
remote2 {
my-dispatcher {
executor = "thread-pool-executor"
type = PinnedDispatcher
}
akka {
actor {
provider = remote
}
remote {
artery {
transport = tcp # See Selecting a transport below
canonical.hostname = "127.0.0.1"
canonical.port = 6062
}
}
}
}
Output:
message count -- 10000 --- time taken - 5871
message count -- 20000 --- time taken - 9043
message count -- 30000 --- time taken - 12198
message count -- 40000 --- time taken - 15363
message count -- 50000 --- time taken - 18649
message count -- 60000 --- time taken - 22074
message count -- 70000 --- time taken - 25487
message count -- 80000 --- time taken - 28820
message count -- 90000 --- time taken - 32118
message count -- 100000 --- time taken - 35634
message count -- 110000 --- time taken - 39146
message count -- 120000 --- time taken - 42539
message count -- 130000 --- time taken - 45997
message count -- 140000 --- time taken - 50013
message count -- 150000 --- time taken - 53466
message count -- 160000 --- time taken - 57117
message count -- 170000 --- time taken - 61246
message count -- 180000 --- time taken - 65051
message count -- 190000 --- time taken - 68809
message count -- 200000 --- time taken - 72908
message count -- 210000 --- time taken - 77091
message count -- 220000 --- time taken - 80855
message count -- 230000 --- time taken - 84679
message count -- 240000 --- time taken - 89089
message count -- 250000 --- time taken - 93132
message count -- 260000 --- time taken - 97360
message count -- 270000 --- time taken - 101442
message count -- 280000 --- time taken - 105656
message count -- 290000 --- time taken - 109665
message count -- 300000 --- time taken - 113706
Total Time - 113707
Is there any wrong I am doing here?. Any observation or suggestion to improve the performance?
The main issue I see with the code is Await.result(). That is a blocking operation, and will most likely affect performance.
I suggest collecting the results in a fixed array / list, use an integer as an array, and consider it complete when the expected number of responses have been received.
I have the following c code, have a large vector and is scattered through the different processor and then gather again, I have a custom MPI type that has been previously tested on another program:
pixel *src = (pixel*) malloc(sizeof(pixel) * MAX_PIXELS);
const int ROOT = 0;
int pid, n_processors;
pixel *receive_buffer;
MPI_Init(NULL, NULL);
MPI_Comm_rank(MPI_COMM_WORLD, &pid);
MPI_Comm_size(MPI_COMM_WORLD, &n_processors);
[...code...]
receive_buffer = (pixel*)malloc(send_count * sizeof(pixel));
//send_count is a value calculated on root and then broadcasted
MPI_Scatter(src, send_count, mpi_pixel_type, receive_buffer, send_count, mpi_pixel_type, ROOT, MPI_COMM_WORLD);
pixel* dst = (pixel*)malloc(sizeof(pixel)*xsize*ysize_per_proccesor);
[...operations with dst and receive buffer...]
MPI_Gather(&receive_buffer, send_count, mpi_pixel_type, src, send_count, mpi_pixel_type, ROOT, MPI_COMM_WORLD);
But this last line is giving me the following error in execution:
[user] Read -1, expected 32769, errno = 14
[user] *** Process received signal ***
[user] Signal: Segmentation fault (11)
[user] Signal code: Address not mapped (1)
[user] Failing at address: 0x7ffe2dd15000
[user] [ 0] /lib/x86_64-linux-gnu/libc.so.6(+0x46210)[0x7f04af898210]
[user] [ 1] /lib/x86_64-linux-gnu/libc.so.6(+0x18e533)[0x7f04af9e0533]
[user] [ 2] /home/yunhi/.openmpi/lib/openmpi/mca_btl_vader.so(+0x3284)[0x7f04ac24c284]
[user] [ 3] /home/yunhi/.openmpi/lib/openmpi/mca_pml_ob1.so(mca_pml_ob1_send_request_schedule_once+0x1c6)[0x7f04ac0add46]
[user] [ 4] /home/yunhi/.openmpi/lib/openmpi/mca_pml_ob1.so(mca_pml_ob1_recv_frag_callback_ack+0x1a9)[0x7f04ac0a6849]
[user] [ 5] /home/yunhi/.openmpi/lib/openmpi/mca_btl_vader.so(mca_btl_vader_poll_handle_frag+0x95)[0x7f04ac24df95]
[user] [ 6] /home/yunhi/.openmpi/lib/openmpi/mca_btl_vader.so(+0x52d7)[0x7f04ac24e2d7]
[user] [ 7] /home/yunhi/.openmpi/lib/libopen-pal.so.40(opal_progress+0x34)[0x7f04af6be0b4]
[user] [ 8] /home/yunhi/.openmpi/lib/openmpi/mca_pml_ob1.so(mca_pml_ob1_send+0x7c5)[0x7f04ac0a1a15]
[user] [ 9] /home/yunhi/.openmpi/lib/libmpi.so.40(ompi_coll_base_gather_intra_linear_sync+0xdf)[0x7f04afc5845f]
[user] [10] /home/yunhi/.openmpi/lib/openmpi/mca_coll_tuned.so(ompi_coll_tuned_gather_intra_dec_fixed+0xb7)[0x7f04ac04da27]
[user] [11] /home/yunhi/.openmpi/lib/libmpi.so.40(PMPI_Gather+0x15a)[0x7f04afc221ea]
[user] [12] ./blurc(+0x2d73)[0x5576edabed73]
[user] [13] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f04af8790b3]
[user] [14] ./blurc(+0x13ce)[0x5576edabd3ce]
[user] *** End of error message ***
I have revised the memories allocations and I can't find why recive_buffer or source is causing address not mapped.
Any suggestions will help, thanks :)
I think you did not mean to pass the address of the receive_buffer point into the function, did you? You want
MPI_Gather(receive_buffer, send_count, mpi_pixel_type, src, send_count, mpi_pixel_type, ROOT, MPI_COMM_WORLD);
My code is to convert a json document into string:
class JMSG_C: public rapidjson::Document
{
inline std::string JMSG_C::get_msg_str()
{
// Convert JSON document to string
rapidjson::StringBuffer buffer;
rapidjson::Writer< rapidjson::StringBuffer > writer(buffer);
Accept(writer);
std::string str = buffer.GetString();
return str;
}
}
Function get_msg_str() fails on segmentation fault. The error message is as in the following. The error does not happen every time but randomly. I suspects that is due to memory allocation failure. But when the program runs, I monitor the system RAM consumption and don't see RAM exhausted. Anybody has any idea?
*** Process received signal ***
Signal: Segmentation fault (11)
Signal code: Address not mapped (1)
Failing at address: 0x96a5e64d
[ 0] [0xb770d40c]
[ 1] /lib/i386-linux-gnu/i686/cmov/libc.so.6(+0x7129e) [0xb732b29e]
[ 2] /lib/i386-linux-gnu/i686/cmov/libc.so.6(+0x73565) [0xb732d565]
[ 3] /lib/i386-linux-gnu/i686/cmov/libc.so.6(__libc_malloc+0x5c) [0xb732fa6c]
[ 4] /home/logia/work/projects/dev_Tao/products/mpitao/mpitao(_ZN9rapidjson12CrtAllocator6MallocEj+0x11) [0x809a7e5]
[ 5] /home/logia/work/projects/dev_Tao/products/mpitao/mpitao(_ZN9rapidjson19MemoryPoolAllocatorINS_12CrtAllocatorEE8AddChunkEj+0x1e) [0x80a4bf6]
[ 6] /home/logia/work/projects/dev_Tao/products/mpitao/mpitao(_ZN9rapidjson19MemoryPoolAllocatorINS_12CrtAllocatorEEC2EjPS1_+0x74) [0x80a2c6a]
[ 7] /home/logia/work/projects/dev_Tao/products/mpitao/mpitao(_ZN9rapidjson8internal5StackINS_19MemoryPoolAllocatorINS_12CrtAllocatorEEEEC2EPS4_j+0x9e)
[ 8] /home/logia/work/projects/dev_Tao/products/mpitao/mpitao(_ZN9rapidjson6WriterINS_19GenericStringBufferINS_4UTF8IcEENS_12CrtAllocatorEEES3_NS_19MemoryPoolAllocatorIS4_EEEC2ERS5_PS7_j+0x2d) [0x809f255]
[ 9] /home/logia/work/projects/dev_Tao/products/mpitao/mpitao(_ZN6JMSG_C11get_msg_strEb+0x64) [0x809ad50]
[10] /home/logia/work/projects/dev_Tao/products/mpitao/mpitao(_Z9cell_mainR6COMM_C+0x7da) [0x80aba23]
[11] /home/logia/work/projects/dev_Tao/products/mpitao/mpitao(main+0xef)
[12] /lib/i386-linux-gnu/i686/cmov/libc.so.6(__libc_start_main+0xe6)
I have a piece of code like this, running on 4 MPI process.
for (i=0;i<niter;i++){
//.. do something with temprs
memcpy(rs, temprs,..) // copy temprs content to rs
MPI_Gather(rs,...0...); //gather result to 0
if (mpiRank == 0) writeToDisk(rs);
}
I want to put 2 last line of code into a function comm_and_save then threaded it so that It can run in parallel with the remaining code, something like below:
boost::thread t1;
for (i=0;i<niter;i++){
//.. do something with temprs
t1.join(); // make sure previous comm_and_save done
memcpy(rs, temprs,..) // copy temprs content to rs
t1 = boost::thread( comm_and_save, rs );
}
However, the code sometime run, sometime hang, sometime throws some error:
local QP operation err (QPN 5a0407, WQE # 00000f02, CQN 280084, index 100677)
[ 0] 005a0407
[ 4] 00000000
[ 8] 00000000
[ c] 00000000
[10] 0270c84f
[14] 00000000
[18] 00000f02
[1c] ff100000
Please enlighten me which part I'm doing incorrectly
Thank you
Use MPI_Init_thread:
http://www.mpi-forum.org/docs/mpi-20-html/node165.htm
and check return status: Available level of thread support
Cheerz.