Akka pre-mature termination - akka

I'm experiencing early termination of the AkkaSystem after first record being read ,entering into the Dead letter without executing the task for all the records
I have 10records in my file , 2 of the records has the matching filename to be pushed to s3.
what could be going wrong here. Please suggest
Sample file record:
xxxxx,ABC,2019-05-10 00:11:00
yyyyyy,XYZ,2019-05-10 00:41:00
import akka.actor.{Actor, ActorSystem, Props}
import scala.io.Source
import scala.sys.process._
class HelloActor extends Actor {
def receive: Receive = {
case line: String => {
// print("Ahshan"+line)
val row = line.split ( "," )
val stdout = new StringBuilder
val stderr = new StringBuilder
val status = Seq ( "/bin/sh", "-c", "ls /Users/ahshan.md/Downloads/".concat ( row ( 2 ).substring ( 0, 10 ) ).concat ( "/*" ).concat ( row ( 0 ) ).concat ( "*" ) ) ! ProcessLogger ( stdout append _, stderr append _ )
// println(status)
// println("stdout: " + stdout)
// println("stderr: " + stderr)
if (status == 0) {
println ( "/bin/sh", "-c", "aws s3 cp ".concat ( stdout.mkString ).concat ( " " ).concat ( "s3://ahshan/".concat ( row ( 1 ) ).concat ( "/" ).concat ( row ( 0 ) ).concat ( ".email" ) ) )
}
else {
// println ( "File Not Found: " + row ( 0 ), row ( 1 ), row ( 2 ).substring ( 0, 10 ) )
println ( "stderr: " + stderr )
}
}
case "finished" => println ( "Hello Ahshan" )
case _ => println ( "Exiting" )
}
}
object AkkaHelloWorld extends App {
// an actor needs an ActorSystem
val system = ActorSystem ( "HelloSystem" )
// create and start the actor
val helloActor = system.actorOf ( Props [HelloActor], name = "helloActor" )
try {
val filename = "/Users/ahshan.md/Downloads/test.txt"
for (line <- Source.fromFile ( filename ).getLines) {
helloActor ! line
}
}
finally {
system.terminate ()
}
}

The ActorSystem and the actors run concurrently with your main thread, sending a message to an actor is async and the sending thread immediately continues, it does not wait for the actor to process the message.
This means that in your code the main thread fires off each line to the actor as fast as it can and then terminates the actor system. In this example it could make sense to move the termination logic into the actor and let the actor terminate the system when it has completed.
That will only work as long as your application is a single actor though, as soon as you add another actor you will have to revisit how you terminate the system.
As an additional note using the process API in Scala like that will block until the process has completed, calling blocking code can have bad consequences, read this section of the docs for more details: https://doc.akka.io/docs/akka/current/typed/dispatchers.html#blocking-needs-careful-management

Related

Wordpress query search 's' exact word

I see this question has been posted multiple times but all of the examples I've tried are not working.
I want to limit the search by keywords to only search for exact word not a phrase or mixed content, so if I search for cheese It need to be limited only for word cheese, right now it will return content with word cheeseburger also.
The part of the search query in my modified template query is right here:
if ( 'featured' === $args['orderby'] ) {
$query_args['orderby'] = [
'menu_order' => 'ASC',
'date' => 'DESC',
'ID' => 'DESC',
];
}
if ( 'rand_featured' === $args['orderby'] ) {
$query_args['orderby'] = [
'menu_order' => 'ASC',
'rand' => 'ASC',
];
}
$job_manager_keyword = sanitize_text_field( $args['search_keywords'] );
if ( ! empty( $job_manager_keyword ) && strlen( $job_manager_keyword ) >= apply_filters( 'job_manager_get_listings_keyword_length_threshold', 2 ) ) {
$query_args['s'] = $job_manager_keyword;
//add_filter( 'posts_search', 'get_job_listings_keyword_search' );
}
$query_args = apply_filters( 'job_manager_get_listings', $query_args, $args );
if ( empty( $query_args['meta_query'] ) ) {
unset( $query_args['meta_query'] );
}
if ( empty( $query_args['tax_query'] ) ) {
unset( $query_args['tax_query'] );
}
/** This filter is documented in wp-job-manager.php */
$query_args['lang'] = apply_filters( 'wpjm_lang', null );
// Filter args.
$query_args = apply_filters( 'get_job_listings_query_args', $query_args, $args );
do_action( 'before_get_job_listings', $query_args, $args );
I can see that $query_args['s'] is standard query for search keywords but some standard query modifications I've tried like this examples
https://wordpress.stackexchange.com/questions/177183/make-a-wp-query-search-match-exactly-the-search-term/17786
WordPress exact search query
Query for exact word search
And other examples too, none of them are working for me.
Anyone have an idea how i can modify it so it can search only for exact word in content?

lauterbach scripting: read peripheral

I am trying to check if a function is called every 2 miliseconds by way of a script. For this I need to read the value of a timer but I get the error 'memory access not allowed' when I try to read it(Data.Long(D:0xF00001010)). This is may code:
GLOBAL &calledFunction &result &difftime &start &end
&calledFunction = "func1"
Go
BREAK.SET &calledFunction /Program
WAIT !STATE.RUN() 5.0s
IF (STATE.RUN())
(
&result = "failed"
)
ELSE
(
&start = Data.Long(D:0xF00001010)
Go
WAIT !STATE.RUN() 5.0s
IF (STATE.RUN())
(
&result = "failed"
)
ELSE
(
&end = Data.Long(D:0xF00001010)
&difftime = &start - &end
IF (&difftime == 2ms)
(
&result = "passed"
)
ELSE
(
&result = "failed"
)
)
)
break.Delete
break
ENDDO
Is there any way I can access the timer's value or maybe another solution to check if the function is called periodically? I tried using the runtime functions but the accuracy is not so good.

AWS SQS Stream Not Shutting Down

On empty receive from AWS SQS I am trying to shutdown and stop listening using below code but it is not happening-
val queueSourceSettings: SqsSourceSettings = SqsSourceSettings(
getQueue(queueType).waitTimeSeconds,
getQueue(queueType).maxBufferSize,
getQueue(queueType).maxBatchSize,
messageAttributeNames = Seq(
MessageAttributeName(TRANSACTION_ID.name)
),
closeOnEmptyReceive = true
)
SqsSource(endpoint.queue.url, endpoint.client, endpoint.queueSourceSettings)
.via(flow)
.recoverWithRetries(-1, {
case e =>
logger.error("Stream Faiure: ", e)
streamFailure(e)
SqsSource(endpoint.queue.url, endpoint.client, endpoint.queueSourceSettings)
.via(flow)
})
.runWith(SqsAckSink(endpoint.queue.url)(ec, endpoint.client))

Perl logging not working on long system call

I have a Perl script that is making a call to the AWS command line to upload a file to S3. It uses Log4Perl to log information about the upload. When the upload is small, a few seconds, the logging works fine -- but when the file is bigger and takes a few minutes then nothing gets logged. The file gets uploaded fine, but the log file is not created.
This is how I'm doing it:
#get start time
my #dt = localtime;
my $TransferStartDate = strftime("%Y-%m-%d", #dt);
my $TransferStartTime = strftime("%H:%M:%S", #dt);
my $aws_command = "/path/to/aws/bin/aws s3 mv $local_file s3://" . $s3_bucket;
my $aws_status = system( $aws_command);
#get stop time
#dt = localtime;
my $TransferEndDate = strftime("%Y-%m-%d", #dt);
my $TransferEndTime = strftime("%H:%M:%S", #dt);
my $log_conf = "
log4perl.rootLogger = $log_level, LOG1
log4perl.appender.LOG1 = Log::Log4perl::Appender::File
log4perl.appender.LOG1.filename = /path/to/logs/aws_s3_log-$TransferEndDate.txt
log4perl.appender.LOG1.mode = append
log4perl.appender.LOG1.layout = Log::Log4perl::Layout::PatternLayout
log4perl.appender.LOG1.layout.ConversionPattern = %d %p %m %n
";
Log::Log4perl::init(\$log_conf);
my $logger = Log::Log4perl->get_logger();
$logger->info( "Transfer command = $aws_command" );
$logger->info( "Transfer start = $TransferStartDate $TransferStartTime" );
$logger->info( "Transfer end = $TransferEndDate $TransferEndTime" );
#check return code
if ( $aws_status == 0 )
{
$logger->info( "Transfer completed successfully");
}
elsif ( $aws_status == 1 )
{
$errstring = "s3 transfer failed status = $aws_status";
$logger->error( $errstring );
}
elsif ( $aws_status == 2 )
{
$errstring = "Syntax error on AWS S3 command status = $aws_status";
$logger->error( $errstring );
}
elsif ( $aws_status == 130 )
{
$errstring = "transfer interrupted by Ctrl-C or other SIGINT status = $aws_status";
$logger->error( $errstring );
}
elsif ( $aws_status == 255 )
{
$errstring = "Command failed status = $aws_status";
$logger->error( $errstring );
}
elsif ( $aws_status == 512 )
{
$errstring = "s3 command formatted incorrectly status = $aws_status";
$logger->error( $errstring );
}
else
{
$errstring = "Error with AWS s3 command status = $aws_status";
$logger->error( $errstring );
}
Any ideas what I am doing wrong? Using system to make the call should cause the Perl script to stop until the upload completes, right?
Since you're logging to file, i would add a $|=1; in the very beginning of the script.
Also i think you should instantiate logger earlier, before launching aws_command, and launch it using ``:
$|=1;
my #dt = localtime;
my $TransferStartDate = strftime("%Y-%m-%d", #dt);
my $TransferStartTime = strftime("%H:%M:%S", #dt);
my $log_conf = "
log4perl.rootLogger = $log_level, LOG1
log4perl.appender.LOG1 = Log::Log4perl::Appender::File
log4perl.appender.LOG1.filename = /path/to/logs/aws_s3_log-$TransferStartDate.txt
log4perl.appender.LOG1.mode = append
log4perl.appender.LOG1.layout = Log::Log4perl::Layout::PatternLayout
log4perl.appender.LOG1.layout.ConversionPattern = %d %p %m %n
";
Log::Log4perl::init(\$log_conf);
my $logger = Log::Log4perl->get_logger();
my $aws_command = "/path/to/aws/bin/aws s3 mv $local_file s3://" . $s3_bucket;
my $aws_output = `$aws_command`;
my $aws_status = $?;
#get stop time
#dt = localtime;
my $TransferEndDate = strftime("%Y-%m-%d", #dt);
my $TransferEndTime = strftime("%H:%M:%S", #dt);
$logger->info( "Transfer command = $aws_command" );
$logger->info( "Transfer result = $aws_result" );
Not sure it'll solve your issue, but might help you narrow your diagnostics.

Screen scraper script won't write to ouptut file

I can't get the Perl script below to write to the file output.html.
I doesn't need to be a CGI script yet, but that is the ultimate intention.
Can anyone tell me why it isn't writing any text to output.html?
#!/usr/bin/perl
#-----------------------------------------------------------------------
# This script should work as a CGI script, if I get it correctly.
# Most CGI scripts for Perl begin with the same line and must be
# stored in your servers cgi-bin directory. (I think this is set by
# your web server.
#
# This scripts will scrape news sites for stories about topics input
# by the users.
#
# Lara Landis
# Sinister Porpoise Computing
# 1/4/2018
# Personal Perl Project
#-----------------------------------------------------------------------
#global_sites = ();
print( "Starting program.\n" );
if ( !( -e "sitedata.txt" ) ) {
enter_site_info( #global_sites );
}
if ( !( -e "scrpdata.txt" ) ) {
print( "scrpdata.txt does not exist. Creating file now.\n" );
print( "Enter the search words you wish to search for below. Press Ctrl-D to finish.\n" );
open( SCRAPEFILE, ">scrpdata.txt" );
while ( $line = <STDIN> ) {
chop( $line );
print SCRAPEFILE ( "$line\n" );
}
close( SCRAPEFILE );
}
print( "Finished getting site data..." );
scrape_sites( #global_sites );
#----------------------------------------------------------------------
# This routine gets information from the user after the file has been
# created. It also has some basic checking to make sure that the lines
# fed to it are legimate domains. This is not an exhaustive list of
# all domains in existence.
#----------------------------------------------------------------------
sub enter_site_info {
my ( #sisites ) = #_;
$x = 1;
open( DATAFILE, ">sitedata.txt" ) || die( "Could not open datafile.\n" );
print( "Enter websites below. Press Crtl-D to finish.\n" );
while ( $x <= #sisites ) {
$sisites[$x] = <STDIN>;
print( "$sisites[$x] added.\n" );
print DATAFILE ( "$sisites[$x]\n" );
$x++;
}
close( DATAFILE );
return #sisites;
}
#----------------------------------------------------------------------
# If the file exists, just get the information from it. Read info in
# from the sites. Remember to create a global array for the sites
# data.
#-----------------------------------------------------------------------
#-----------------------------------------------------------------------
# Get the text to find in the sites that are being scraped. This requires
# nested loops. It starts by going through the loops for the text to be
# scraped, and then it goes through each of the websites listend in the
# sitedata.txt file.
#-----------------------------------------------------------------------
sub scrape_sites {
my ( #ss_info ) = #_;
#gsi_info = ();
#toscrape = ();
$y = 1;
#---------------------------
# Working code to be altered
#---------------------------
print( "Getting site info..." );
$x = 1;
open( DATAFILE, "sitedata.txt" ) || die( "Can't open sitedata.txt.txt\n" );
while ( $gsi_info[$x] = <DATAFILE> ) {
chop( $gsi_info[$x] );
print( "$gsi_info[$x]\n" );
$x++;
}
close( DATAFILE );
open( SCRAPEFILE, "scrpdata.txt" ) || die( "Can't open scrpdata.txt\n" );
print( "Getting scrape data.\n" );
$y = 1;
while ( $toscrape[$y] = <SCRAPEFILE> ) {
chop( $toscrape[$y] );
$y++;
}
close( SCRAPEFILE );
print( "Now opening the output file.\n" );
$z = 1;
open( OUTPUT, ">output.html" );
print( "Now scraping sites.\n" );
while ( $z <= #gsi_info ) { #This loop contains SITES
system( "rm -f index.html.*" );
system( "wget $gsi_info[$z]" );
$z1 = 1;
print( "Searching site $gsi_info[$z] for $toscrape[$z1]\n" );
open( TEMPFILE, "$gsi_info[$z]" );
$comptext = <TEMPFILE>;
while ( $comptext =~ /$toscrape[z1]/ig ) { # This loop fetches data from the search terms
print( "Now scraping $gsi_info[$z] for $toscrape[$z1]\n" );
print OUTPUT ( "$toscrape[$z1]\n" );
$z1++;
}
close( TEMPFILE );
$z++;
}
close( OUTPUT );
return ( #gsi_info );
}
You're making assumptions about the current work directory that are often incorrect. You seem to assume the current work directory is the directory in which the script resides, but that's never guaranteed, and it's often / for CGI scripts.
"sitedata.txt"
should be
use FindBin qw( $RealBin );
"$RealBin/sitedata.txt"
There could also be a permission error. You should include the error cause ($!) in your error message when open fails so you know what is causing the problem!
While you're checking some, you're not checking all of your open or system calls. If they fail, the program will keep going without an error message telling you why.
You can add checks to all of these, but it's easy to forget. Instead, use autodie to do the checks for you.
You'll also want to use strict to ensure you haven't made any variable typos, and use warnings to warn you about small mistakes. See this answer for more.
Also #global_sites is empty so enter_site_info() isn't going to do anything. And scrape_sites() does nothing with its argument, #ss_info.
All of these things are helpful. Thank you. I found the problem. I was opening the wrong file. It was putting the error-checking in on the file that let me spot the error. It should have been
open (TEMPFILE, "index.html") || die ("Cannot open index.html\n");
I have taken as many of the suggestions as I remembered and included them in the code. I still need to implement the directory advice, but it should not be difficult.