To run updateHITs.sh, I have to use the HIT ID and hittypeid. I have the ID, but I'm not sure how to get the type from command line tools. Any ideas?
There are two places you can get a HITTypeID using the Command Line Tools. First, it's shown when you create HITs. For example, here's the output run executing run.sh in the helloworld sample:
0088653dacfc:helloworld jrb$ ./run.sh
Unable to find a $JAVA_HOME at "/usr", continuing with system-provided Java...
--[Initializing]----------
Input: ../samples/helloworld/helloworld.input
Properties: ../samples/helloworld/helloworld.properties
Question File: ../samples/helloworld/helloworld.question
Preview mode disabled
--[Loading HITs]----------
Start time: Thu Mar 31 17:01:16 PDT 2016
Created HIT 1: HITId=3EHVO81VN5LKU5SFUDTKRPRS0ZQ1HA
Created HIT 2: HITId=3FBEFUUYRK50R4LYUDV95ZV5VFG6A2
Created HIT 3: HITId=307L9TDWJYS8E4S5KF2GK23LKVD3ND
Created HIT 4: HITId=35ZRNT9RUIYQWPSHLU2TZR3QA5I3OS
Created HIT 5: HITId=3X55NP42EOG8HP4I0UF8OR8GKAE3P9
You may see your HIT(s) with HITTypeId '3PFYYPJUJW5X02IODV4MGVYUBJX2Z0' here:
https://workersandbox.mturk.com/mturk/preview?groupId=3PFYYPJUJW5X02IODV4MGVYUBJX2Z0
End time: Thu Mar 31 17:01:17 PDT 2016
--[Done Loading HITs]----------
Total load time: 1 seconds.
Successfully loaded 5 HITs.
Also, you can get the HITTypeId in the .success file that's produced when you execute the run.sh command (to create HITs). For example, after I ran the above command, I had a file called helloworld.success in the same folder with the contents of (note I added a few tabs to make it look pretty, but don't try and use this directly in your file, the extra tabs mean it'll probably not work):
hitid hittypeid
3EHVO81VN5LKU5SFUDTKRPRS0ZQ1HA 3PFYYPJUJW5X02IODV4MGVYUBJX2Z0
3FBEFUUYRK50R4LYUDV95ZV5VFG6A2 3PFYYPJUJW5X02IODV4MGVYUBJX2Z0
307L9TDWJYS8E4S5KF2GK23LKVD3ND 3PFYYPJUJW5X02IODV4MGVYUBJX2Z0
35ZRNT9RUIYQWPSHLU2TZR3QA5I3OS 3PFYYPJUJW5X02IODV4MGVYUBJX2Z0
3X55NP42EOG8HP4I0UF8OR8GKAE3P9 3PFYYPJUJW5X02IODV4MGVYUBJX2Z0
The updateHITs.sh file is expecting a .success file as input. If all you have is a hitid, there's actually a sneaky way to do this. It turns out that you don't actually need a hittypeid in the .success file for it to work. So paste the code into a file, name it something like sneaky.success:
hitid
3FBEFUUYRK50R4LYUDV95ZV5VFG6A2
And then run the command:
./updateHITs.sh -success sneaky.success -properties new_hit.properties
Where "new_hit.properties" contains all the properties you want to update these HITs to.
Related
Summary of the problem
I've setup a repository with the checks I developed, designed to be used exclusively in an another repository.
The problem I've been facing is that when I run pre-commit run -a -v the checks don't go through every file, and time to time they even change!
A little more details
I've run the [identity][1] check and it prints every file in the repo, meaning the files are read by pre-commit (per my understanding)
When I execute pre-commit run the file in staging are correctly parsed
Why do i think that the checks don't run on every file?
I always print something to the standard output for every check, and when I run it verbosely it only prints 5 to 7 lines, meaning it checked only those.
If I try and edit a file (breaking a check) whose not on the short list, the check still passes through
Some code
The structure of the folder containing the files to be checked
./
articles/
some_name/
file.md
file.png
and_so_on.jpeg
team/
some_other_name/
file.md
propic.jpg
Summary of the .pre-commit-hooks.yaml
- id: team-check-name
name: blabla
description: blabla
entry: team-checkname
files: 'team/.*/'
types: [markdown]
language: python
- id: article-check-name
name: blabla
description: blabla
entry: article-checkname
files: 'articles/.*/'
types: [markdown]
language: python
Essentially the checks supposed to run on articles/.*/ start with article- and they all have this property: files: 'articles/.*/'.
Similarly every check supposed to run on team/.*/ starts with team- and they all have files: 'team/.*/'
Summary of the .pre-commit-config.yaml
repos:
- repo: https://github.com/repourl
rev: commit_hash
hooks:
- id: team-check-name
- id: article-check-name
But as you can see, I don't override any settings so it shouldn't interfere
the hook tools you've written are incorrect -- they only process sys.argv[1] rather than positional arguments
your code uses a lot of global variables so it's not a straightforward refactor -- typically you'd either use argparse to collect nargs='*' or loop over sys.argv[1:] (if you don't have any options)
as to why this is the convention -- it's very wasteful to start a linter process over and over to lint a single file (often times executable startup cost dwarfs the actual linting / formatting process)
disclaimer: I wrote pre-commit
I am trying to debug a very sporadic issue in my application. If ran ~1000 times my application surely hits a certain line it shouldn't and I would like to view the stack.
I tried using a gdb script cmd.gdb for this:
set logging overwrite on
set pagination off
set $n = 1000
break file.c:496
while $n-- > 0
ignore 1 9
condition 1 global_var == 10
run
end
How should I modify this script in order to print the stack when the breakpoint is reached?
I tried adding this after "run":
if $_siginfo
bt
loop_break
end
but it doesn't seem to work.
Actually, I have a Github repo with a Python-GDB extension, which does exactly the same thing as You have described, but with some more functionality.
You can just clone the repo:
git clone https://github.com/Viaceslavus/gdb-debug-until.git
and feed the python script to GDB with the following command inside GDB:
source <python script path>
Then, according to your example, you should run the next command:
debug-until file.c:496 --args="" --var-eq="global_var:10" -r=1000
*some remarks:
file.c:496 here is a starting breakpoint
"--args" parameter contains the arguments for your program
"--var-eq" is a debugging event, where 'global_var' is a variable name and '10' is a value
and finally the "-r" option specifies the number of times the program will be ran.
So all together this command will run your program 1000 times and will immediately notify You when the 'global_var' will be equal to 10.
Any additional information about the project could be found here:
https://github.com/Viaceslavus/gdb-debug-until.git in the README file.
I'm trying to run a single test class ThisIsTheClassTest.java in a Java Play project but fails and cannot seem to find the cause.
When I run test:compile the response is plain [success] Total time: 1 s, completed Jun 5, 2018 9:30:19 PM
But when using autocomplete to list all available test classes there is one missing.
It seems that it might be the filename itself (for whatever reason) as it all works well if I copy the entire code into another file ThisIsNotTheFile.java and run test:Compile. But even if I delete the entire file ThisIsTheClassTest.java and add it again it does not work.
If one of the classes did not compile I would not expect the answer [success].
Is there a way to see skipped/failed/ignored compilations or at least additional logging?
The code is probably in the wrong directory.
I'm now using google glog. When I'm debugging the program, every time the process restart, a new log file generated, identified by the new pid as filename's sufix, like this
ied_config.20131220-142934.4524
ied_config.20131220-171227.3948
ied_config.20131225-170117.7032
ied_config.20131225-170131.5200
ied_config.20131225-171450.7820
ied_config.20131225-172336.4116
ied_config.20131225-172924.6120
ied_config.20131225-173000.2980
ied_config.20131225-173037.1960
ied_config.20131225-173144.7304
ied_config.20131226-095843.1012
smv_client.20131219-082120.7184
smv_client.20131219-162339.5508
smv_client.20131219-163055.6156
smv_client.20131219-163155.4892
smv_client.20131219-163206.5576
smv_client.20131219-163216.6340
smv_client.20131219-163236.6952
smv_client.20131219-163307.7940
smv_client.20131219-163317.4920
smv_client.20131219-163347.6556
smv_client.20131219-163408.5124
smv_client.20131219-163428.2644
smv_client.20131219-163448.6040
smv_client.20131219-163529.6948
smv_client.20131219-163539.1592
smv_client.20131219-163549.3776
smv_client.20131219-172949.5412
smv_client.20131219-173000.4180
smv_client.20131219-173010.7432
smv_client.20131220-170628.636
smv_client.20131220-170930.3904
smv_client.20131226-095841.1296
I want to join these log just into one file for each program, I can't file the configuration for the glog ,any helps?
If you want to change the name of log files, you may need to call SetLogDestination():
google::SetLogDestination(google::INFO, "/var/tmp/another_destination.INFO");
From release 0.5 on May 8, 2021
Allow a log file to be simply named "foobar.log" with no appended
string #124
google::SetLogDestination(google::INFO, "/var/tmp/another_destination.INFO");
FLAGS_timestamp_in_logfile_name = false;
I'm running several simulations using Condor and have coded the program so that it outputs a progress status in the console. This is done at the end of a loop where it simply prints the current time (this can also be percentage or elapsed time). The code looks something like this:
printf("START");
while (programNeedsToRum) {
// Run code repetitive code...
// Print program status update
printf("[%i:%i:%i]\r\n", hours, minutes, seconds);
}
printf("FINISH");
When executing normally (i.e. in the terminal/cmd/bash) this works fine, but the condor nodes don't seem to printf() the status. Only once the simulation has finished, all the status updates have been outputted to the file but then it's no longer of use. My *.sub file that I submit to condor looks like this:
universe = vanilla
executable = program
output = out/out-$(Process)
error = out/err-$(Process)
queue 100
When submitted the program executes (this is confirmed in condor_q) and the output files contain this:
START
Only once the program has finished running its corresponding output file shows (example):
START
[0:3:4]
[0:8:13]
[0:12:57]
[0:18:44]
FINISH
Whilst the program executes, the output file only contains the START text. So I came to the conclusion that the file is not updated if the node executing program is busy. So my question is, is there a way of updating the output files manually or gather any information on the program's progress in a better way?
Thanks already
Max
What you want to do is use the streaming output options. See the stream_error and stream_output options you can pass to condor_submit as outlined here: http://research.cs.wisc.edu/htcondor/manual/current/condor_submit.html
By default, HTCondor stores stdout and stderr locally on the execute node and transfers them back to the submit node on job completion. Setting stream_output to TRUE will ask HTCondor to instead stream the output as it occurs back to the submit node. You can then inspect it as it happens.
Here's something I used a few years ago to solve this problem. It uses condor_chirp which is used to transfer files from the execute host to the submitter. I have a python script that executes the program I really want to run, and redirects its output to a file. Then, periodically, I send the output file back to the submit host.
Here's the Python wrapper, stream.py:
#!/usr/bin/python
import os,sys,time
os.environ['PATH'] += ':/bin:/usr/bin:/cygdrive/c/condor/bin'
# make sure the file exists
open(sys.argv[1], 'w').close()
pid = os.fork()
if pid == 0:
os.system('%s >%s' % (' '.join (sys.argv[2:]), sys.argv[1]))
else:
while True:
time.sleep(10)
os.system('condor_chirp put %s %s' % (sys.argv[1], sys.argv[1]))
try:
os.wait4(pid, os.WNOHANG)
except OSError:
break
And my submit script. The problem ran sh hello.sh, and redirected the output to myout.txt:
universe = vanilla
executable = C:\cygwin\bin\python.exe
requirements = Arch=="INTEL" && OpSys=="WINNT60" && HAS_CYGWIN==TRUE
should_transfer_files = YES
transfer_input_files = stream.py,hello.sh
arguments = stream.py myout.txt sh hello.sh
transfer_executable = false
It does send the output in its entirety, so take that in to account if you have a lot of jobs running at once. Currently, its sending the output every 10 seconds .. you may want to adjust that.
with condor_tail you can view the output of a running process.
to see stdout just add the job-ID (and -f if you want to follow the output and see the updates immediately. Example:
condor_tail 314.0 -f