Extract "Urls" FROM Json-Data with Imacros - imacros

I'm using iMacros for Firefox and want to extract some urls from a JSON file
I Tried to extract these urls
http://www.google.com,
http://www.yahoo.com,
http://www.amazon.com
The JSON file looks like this:
{
"names": {
"name": "websites"
},
"urls": [
{
"id": 10,
"url": "http://www.google.com"
},
{
"id": 11,
"url": "http://www.yahoo.com"
},
{
"id": 12,
"url": "http://www.amazon.com"
}
]
}
My IMACROS CODE :
VERSION BUILD=8601111 RECORDER=FX
TAB T=1
SET !ENCRYPTION NO
WAIT SECONDS=1
TAG POS=1 TYPE=TD ATTR=TXT:url EXTRACT=TXT
TAG POS=2 TYPE=TD ATTR=TXT:url EXTRACT=TXT
TAG POS=3 TYPE=TD ATTR=TXT:url EXTRACT=TXT
SAVEAS TYPE=EXTRACT FOLDER=* FILE=links.txt
Thanks .


Try to play this macro:
SET !EXTRACT_TEST_POPUP NO
TAG POS=1 TYPE=* ATTR=* EXTRACT=TXT
SET !EXTRACT EVAL("var u = []; var a = JSON.parse('{{!EXTRACT}}').urls; for (i in a) u.push(a[i].url); u.join('[EXTRACT]');")
SAVEAS TYPE=EXTRACT FOLDER=* FILE=links.txt

Related

How to add text "Before" Extracted Text?

I tried to collect data myself on social media, and I want to add text before the extract results.
Hire My Code :
VERSION BUILD=9030808 RECORDER=FX
SET !ERRORIGNORE YES
SET !EXTRACT_TEST_POPUP NO
TAB T=1
TAG POS=1 TYPE=STRONG ATTR=TXT:* EXTRACT=TXT
SAVEAS TYPE=EXTRACT FOLDER=C:\test FILE=extract.txt
TAG POS=1 TYPE=DIV ATTR=TXT:/* EXTRACT=TXT
SET !EXTRACT EVAL("var s = '{{!EXTRACT}}'.replace(/\\//, ''); s;")
SAVEAS TYPE=EXTRACT FOLDER=C:\test FILE=extract.txt
TAG POS=1 TYPE=SPAN ATTR=TXT:*#*.* EXTRACT=TXT
SAVEAS TYPE=EXTRACT FOLDER=C:\test FILE=extract.txt
TAG POS=1 TYPE=SPAN ATTR=class:_c24<SP>_2ieq EXTRACT=TXT
SET !EXTRACT EVAL("var s = '{{!EXTRACT}}'.replace(/Birth \\date/, ''); s;")
SAVEAS TYPE=EXTRACT FOLDER=C:\test FILE=extract.txt
I use this code to remove Special Character ̶"/̶"
SET !EXTRACT EVAL("var s = '{{!EXTRACT}}'.replace(/\\//, ''); s;")
I use this code because i want to remove unwanted text "B̶i̶r̶t̶h̶ ̶d̶a̶t̶e̶ 12 April 1992"
SET !EXTRACT EVAL("var s = '{{!EXTRACT}}'.replace(/Birth \\date/, ''); s;")
the results of the code :
Hilda
100992837
hilda8872#gmail.com
12 April 1992
Now... I want to add text to the results like this:
Name : Hilda
User ID : 100992837
Email : hilda8872#gmail.com
Date of Birth : 12 April 1992
What should I do ?
which one should i change?
please, this is my college assignment.

I tried use ADD !EXTRACT ..., but the result contains special characters.
ADD !EXTRACT Name<SP>:<SP>
TAG POS=1 TYPE=STRONG ATTR=TXT:* EXTRACT=TXT
SAVEAS TYPE=EXTRACT FOLDER=C:\test FILE=extract.txt
ADD !EXTRACT User<SP>ID<SP>:<SP>
TAG POS=1 TYPE=DIV ATTR=TXT:/* EXTRACT=TXT
SET !EXTRACT EVAL("var s = '{{!EXTRACT}}'.replace(/\\//, ''); s;")
SAVEAS TYPE=EXTRACT FOLDER=C:\test FILE=extract.txt
ADD !EXTRACT Email<SP>:<SP
TAG POS=1 TYPE=SPAN ATTR=TXT:*#*.* EXTRACT=TXT
SAVEAS TYPE=EXTRACT FOLDER=C:\test FILE=extract.txt
ADD !EXTRACT Date<SP>of<SP>Birth<SP>:<SP>
TAG POS=1 TYPE=SPAN ATTR=class:_c24<SP>_2ieq EXTRACT=TXT
SET !EXTRACT EVAL("var s = '{{!EXTRACT}}'.replace(/Birth \\date/, ''); s;")
SAVEAS TYPE=EXTRACT FOLDER=C:\test FILE=extract.txt
Result :
"Name : ","Hilda"
"User ID : ","100992837"
"Email : ","hilda8872#gmail.com"
"Date of Birth : ","12 April 1992"

SOLVED by Add this Code :
TAG POS=1 TYPE=STRONG ATTR=TXT:* EXTRACT=TXT
TAG POS=1 TYPE=INPUT:TEXT FORM=ACTION:/search/ ATTR=NAME:query CONTENT=Nama<SP>:<SP>{{!EXTRACT}}
SET !EXTRACT NULL
TAG POS=1 TYPE=INPUT:TEXT FORM=ACTION:/search/ ATTR=TXT:* EXTRACT=TXT
SAVEAS TYPE=EXTRACT FOLDER=C:\Users\CPS\Desktop FILE=extract.txt
Result :
Name : Hilda

Why can not download the captcha using the following code

I was just trying to donwload the captcha from one website everything with educational purposes. But when i use the ONDOWNLOAD command, the captcha image never is in the tmp FOLDER.
So what im doing WRONG. I tried using Disabling error notifications, but still never the image is downloaded into the folder.
URL GOTO=https://www.coinpayments.net/index.php?ref=fe31773be1c8558db56c7b9f3063ae08
TAG POS=1 TYPE=A ATTR=TXT:Sign<SP>Up
TAG POS=2 TYPE=DIV ATTR=TXT:Create<SP>New<SP>Account<SP>Please<SP>make<SP>it<SP>at<SP>le*
'SEARCH SOURCE=REGEXP:\\b[A-Z0-9._%+-]+#[A-Z0-9.-]+\\.[A-Z]{2,4}\\b EXTRACT=$1
TAG POS=1 TYPE=INPUT:TEXT FORM=ACTION:register ATTR=NAME:username CONTENT=examplethenomtest
TAG POS=1 TYPE=INPUT:EMAIL FORM=ACTION:register ATTR=NAME:email1 CONTENT={{!EXTRACT}}
TAG POS=1 TYPE=INPUT:EMAIL FORM=ACTION:register ATTR=NAME:email2 CONTENT={{!EXTRACT}}
SET !ENCRYPTION NO
TAG POS=1 TYPE=INPUT:PASSWORD FORM=ACTION:register ATTR=NAME:pass1 CONTENT=k1k1k1k1
TAG POS=1 TYPE=INPUT:PASSWORD FORM=ACTION:register ATTR=NAME:pass2 CONTENT=k1k1k1k1
TAG POS=1 TYPE=INPUT:CHECKBOX FORM=ACTION:register ATTR=ID:checkbox CONTENT=YES
TAG POS=2 TYPE=INPUT:CHECKBOX FORM=ACTION:register ATTR=ID:checkbox CONTENT=YES
SET url https://www.coinpayments.net/index.php?ref=fe31773be1c8558db56c7b9f3063ae08
SET apikey apikey
SET path C:\tmp\
SET file freeditcoin.jpg
SET tw 5
TAB OPEN
'TAB T=1
PROXY ADDRESS=0.0.0.0:0
'TAB CLOSEALLOTHERS
SET !TIMEOUT_PAGE 10
'URL GOTO={{url}}
SET !ENCRYPTION NO
SET !VAR1 EVAL("var randomNumber=Math.floor(Math.random()*10 + 1); randomNumber;")
Wait Seconds={{!Var1}}
FILEDELETE NAME={{path}}{{file}}
ONDOWNLOAD FOLDER={{path}} FILE={{file}}
TAG POS=1 TYPE=IMG ATTR=ID:captcha
WAIT SECONDS=11
SET !VAR1 EVAL("var randomNumber=Math.floor(Math.random()*10 + 1); randomNumber;")
Wait Seconds={{!Var1}}
WAIT SECONDS=6
TAG POS=1 TYPE=DIV ATTR=ID:captcha CONTENT=EVENT:SAVE_ELEMENT_SCREENSHOT
WAIT SECONDS=6

I checked the code you gave here is the list of issues.
The line having ONDOWNLOAD should be immediately followed by download of the captcha, so it should be give as follows.
SET !VAR1 EVAL("var randomNumber=Math.floor(Math.random()*10 + 1); randomNumber;")
Wait Seconds={{!Var1}}
ONDOWNLOAD FOLDER={{path}} FILE={{file}}
TAG POS=1 TYPE=IMG ATTR=ID:captcha CONTENT=EVENT:SAVE_ELEMENT_SCREENSHOT
Also notice that in the above code, the last line you were giving the type attribute as IMG instead of DIV.
Regarding the variable path, please give some other folder other than the C Drive which may be the root of your computer, try giving some path like C:\Users\asdf\Desktop. Using such a path, I can see that the image is getting downloaded!
SET path D:\
Below is the complete code, please let me know if your issues is resolved completely!
URL GOTO=https://www.coinpayments.net/index.php?ref=fe31773be1c8558db56c7b9f3063ae08
TAG POS=1 TYPE=A ATTR=TXT:Sign<SP>Up
TAG POS=2 TYPE=DIV ATTR=TXT:Create<SP>New<SP>Account<SP>Please<SP>make<SP>it<SP>at<SP>le*
'SEARCH SOURCE=REGEXP:\\b[A-Z0-9._%+-]+#[A-Z0-9.-]+\\.[A-Z]{2,4}\\b EXTRACT=$1
TAG POS=1 TYPE=INPUT:TEXT FORM=ACTION:register ATTR=NAME:username CONTENT=examplethenomtest
TAG POS=1 TYPE=INPUT:EMAIL FORM=ACTION:register ATTR=NAME:email1 CONTENT={{!EXTRACT}}
TAG POS=1 TYPE=INPUT:EMAIL FORM=ACTION:register ATTR=NAME:email2 CONTENT={{!EXTRACT}}
SET !ENCRYPTION NO
TAG POS=1 TYPE=INPUT:PASSWORD FORM=ACTION:register ATTR=NAME:pass1 CONTENT=k1k1k1k1
TAG POS=1 TYPE=INPUT:PASSWORD FORM=ACTION:register ATTR=NAME:pass2 CONTENT=k1k1k1k1
TAG POS=1 TYPE=INPUT:CHECKBOX FORM=ACTION:register ATTR=ID:checkbox CONTENT=YES
TAG POS=2 TYPE=INPUT:CHECKBOX FORM=ACTION:register ATTR=ID:checkbox CONTENT=YES
SET url https://www.coinpayments.net/index.php?ref=fe31773be1c8558db56c7b9f3063ae08
SET apikey apikey
SET path D:\
SET file freeditcoin.jpg
SET tw 5
TAB OPEN
'TAB T=1
PROXY ADDRESS=0.0.0.0:0
'TAB CLOSEALLOTHERS
SET !TIMEOUT_PAGE 10
'URL GOTO={{url}}
SET !ENCRYPTION NO
SET !VAR1 EVAL("var randomNumber=Math.floor(Math.random()*10 + 1); randomNumber;")
Wait Seconds={{!Var1}}
ONDOWNLOAD FOLDER={{path}} FILE={{file}}
TAG POS=1 TYPE=IMG ATTR=ID:captcha CONTENT=EVENT:SAVE_ELEMENT_SCREENSHOT

Looping and text extracting with iMacro

I am trying to extract some text from the web with iMacro.
I use Chrome and Firefox and latest iMacros for these. I have looked the ways from google and tried to put it to loop, but somehow it's not working still. I can extract tagging them one by one with names and all the things but would like to set it to loop.
I tried to use
SET !Loop 1
TAG POS=1 TYPE=DIV ATTR=* EXTRACT=TXT
also
TAG POS={{!Loop}} TYPE=DIV ATTR=* EXTRACT=TXT
but it's not working
My initial code is here, I appreciate a lot if someone could point out where to put the loop, so it works how to make it work:
VERSION BUILD=9030808 RECORDER=FX
TAB T=1
URL GOTO=https://angel.co/blogging-platforms
SET !EXTRACT_TEST_POPUP NO
'Extract
TAG POS=1 TYPE=DIV ATTR=TXT:MessageParty EXTRACT=TXT
TAG POS=1 TYPE=DIV ATTR=TXT:Chat<SP>with<SP>People<SP>Near<SP>You
EXTRACT=TXT
TAG POS=1 TYPE=DIV
ATTR=TXT:New<SP>York<SP>City<SP>·<SP>Location<SP>Based<SP>Services
EXTRACT=TXT
TAG POS=1 TYPE=DIV ATTR=TXT:Apr<SP>’12 EXTRACT=TXT
TAG POS=1 TYPE=DIV ATTR=TXT:23 EXTRACT=TXT
TAG POS=32 TYPE=DIV ATTR=TXT: EXTRACT=TXT
TAG POS=33 TYPE=DIV ATTR=TXT: EXTRACT=TXT
'Save as
SAVEAS TYPE=EXTRACT FOLDER=* FILE=platforms.csv
Also how I can set it so that it saves all extracted data on the different rows. When I tried it with tagging 5 and running the macro then it saved all on the same line to different columns.
If I remove ATTR=TXT:MessageParty and replace it with the wildcard ATTR=* then it doesn't work either.
Thank you a lot for your help!

Are you looking for the code like suggested below?
TAG POS={{!LOOP}} TYPE=DIV ATTR=class:name EXTRACT=TXT
TAG POS={{!LOOP}} TYPE=DIV ATTR=class:blurb EXTRACT=TXT
TAG POS={{!LOOP}} TYPE=DIV ATTR=class:tags EXTRACT=TXT
' and so on '
SAVEAS TYPE=EXTRACT FOLDER=* FILE=platforms.csv
I go manually to https://angel.co/blogging-platforms and run the macro above.
It gives me the 'platforms.csv' file like this:
"Squarespace","Build it beautiful.","New York City · Blogging Platforms"
"MessageParty","Chat with People Near You","New York City · Location Based Services"
"Medium","We make Medium: A Publishing Platform.","San Francisco · Blogging Platforms"
"Svbtle","A new kind of publishing platform.","San Francisco · Blogging Platforms"
"Posterous","","San Francisco · Networking"
"Six Apart","","Tokyo · Blogging Platforms"
"Automattic","","San Francisco · Forums"
etc.
(I use 'iMacros for Firefox' v.8.9.7)
Adding the code for the 'Joined' and 'Followers' fields:
TAG POS={{!LOOP}} TYPE=DIV ATTR=class:name EXTRACT=TXT
TAG POS={{!LOOP}} TYPE=DIV ATTR=class:blurb EXTRACT=TXT
TAG POS={{!LOOP}} TYPE=DIV ATTR=class:tags EXTRACT=TXT
TAG POS=R1 TYPE=DIV ATTR=class:"value" EXTRACT=TXT
TAG POS=R1 TYPE=DIV ATTR=class:"value" EXTRACT=TXT
SAVEAS TYPE=EXTRACT FOLDER=* FILE=platforms.csv

iMacros to extract google results

I'm trying to extract google search results using iMacros, it does extracts the URL but instead of saving it redirected to extracted URL.
VERSION BUILD=10022823
TAB T=1
TAB CLOSEALLOTHERS
URL GOTO=https://www.google.com/?gws_rd=ssl
TAG POS=1 TYPE=INPUT:TEXT FORM=NAME:f ATTR=NAME:q CONTENT=Jhony
TAG POS=1 TYPE=BUTTON:SUBMIT FORM=NAME:f ATTR=NAME:btnG
'text input activated
TAG POS=1 TYPE=A ATTR=ONMOUSEDOWN:return* EXTRACT=HREF
SAVEAS TYPE=EXTRACT FOLDER=* FILE=links.csv
PROMPT {{!EXTRACT}}
SET !EXTRACT NULL
can anyone suggest what I'm doing wrong?

Perhaps, this could help:
TAB T=1
TAB CLOSEALLOTHERS
URL GOTO=https://www.google.com/?gws_rd=ssl
TAG POS=1 TYPE=INPUT:TEXT FORM=NAME:f ATTR=NAME:q CONTENT=Jhony
TAG POS=1 TYPE=BUTTON:SUBMIT FORM=NAME:f ATTR=NAME:btnG
TAG POS=1 TYPE=H3 ATTR=CLASS:"r" EXTRACT=HTM
SET !EXTRACT EVAL("'{{!EXTRACT}}'.match(/a href=\"(.*)\" onmousedown=/)[1];")
SAVEAS TYPE=EXTRACT FOLDER=* FILE=links.csv

Imacros script stopped working all of a sudden

I have written this code earlier. Previously it was working fine. But suddenly one day iMacros stopped working without I changed anything....
it stops at line 19
TAG POS=1 TYPE=INPUT:SUBMIT FORM=ACTION:http://api.deathbycaptcha.com/decaptcher ATTR=VALUE:Send
Even when I set the IGNOREERROR YES
Please help
VERSION BUILD=8300326 RECORDER=FX
TAB T=1
SET !EXTRACT_TEST_POPUP NO
SET !ERRORIGNORE YES
SET !LOOP 1
URL GOTO=http://bootleggers.us/jail.php
ONDOWNLOAD FOLDER=/Users/dymphi/Desktop/ FILE=image.jpg WAIT=YES
TAG POS=1 TYPE=IMG ATTR=HREF:http://www.google.com/recaptcha/api/image* CONTENT=EVENT:SAVEITEM
TAB OPEN
TAB T=2
URL GOTO=api.deathbycaptcha.com/decaptcher?function=picture2&print_format=html
FRAME F=0
TAG POS=1 TYPE=INPUT:TEXT FORM=ACTION:http://api.deathbycaptcha.com/decaptcher ATTR=NAME:username CONTENT=dbcname
TAG POS=1 TYPE=INPUT:TEXT FORM=ACTION:http://api.deathbycaptcha.com/decaptcher ATTR=NAME:password CONTENT=dbcpassword
TAG POS=1 TYPE=INPUT:FILE FORM=ACTION:http://api.deathbycaptcha.com/decaptcher ATTR=NAME:pict CONTENT=/Users/dymphi/Desktop/image.jpg
TAG POS=1 TYPE=INPUT:SUBMIT FORM=ACTION:http://api.deathbycaptcha.com/decaptcher ATTR=VALUE:Send
TAG POS=6 TYPE=TD ATTR=* EXTRACT=TXT
SET !VAR1 {{!EXTRACT}}
TAB CLOSE
TAB T=1
FRAME F=0
TAG POS=1 TYPE=INPUT:TEXT FORM=ACTION:/jail.php ATTR=ID:recaptcha_response_field CONTENT={{!VAR1}}
TAG POS=1 TYPE=INPUT:SUBMIT FORM=ACTION:/jail.php ATTR=VALUE:Continueplaying!
URL GOTO=http://bootleggers.us/jail.php
TAG POS=1 TYPE=A ATTR=TXT:Bustout!
filedelete name=/Users/dymphi/Desktop/image.jpg
WAIT SECONDS=110

I'm also using deathbycaptcha for solving captcha and my code works fine, you should set a timeout before submitting:
URL GOTO=http://api.deathbycaptcha.com/decaptcher?function=picture2&print_format=html
TAG POS=1 TYPE=INPUT ATTR=NAME:username CONTENT=username
TAG POS=1 TYPE=INPUT ATTR=NAME:password CONTENT=pass
TAG POS=1 TYPE=INPUT ATTR=NAME:pict CONTENT=/Users/dymphi/Desktop/image.jpg
SET !TIMEOUT_STEP 30
TAG POS=1 TYPE=INPUT ATTR=TYPE:submit
TAG POS=6 TYPE=TD ATTR=* EXTRACT=TXT
TAB CLOSE
TAB T=1
SET !VAR1 {{!EXTRACT}}
I tested it even when the image doesn't exist, my version of iMacros is 8.3.0. Hope it will work.