New to Python and have yet to use or come across __builtin__ and was curious about what the following code is actually doing:
import sys
try:
import __builtin__
input = getattr(__builtin__, 'raw_input')
except (ImportError, AttributeError):
pass
username = input("username: ")
password = input("password: ")
Is this code being used to basically check whether or not the script is being run with Python 2 or Python 3 and if it's Python 2, input() is converted to raw_input()?
This is rather poorly-written code to make input() refer to raw_input() if the latter exists. That makes it compatible with both Python 2 and Python 3. A simpler way to do it would be this:
try:
input = raw_input
except NameError:
pass
username = input("username: ")
password = input("password: ")
The __builtin__ module is the module where all the builtin objects like input() and raw_input() live. But we don't need it in this case. In Python 3.x, this is called builtins, which is why the author of this code is catching ImportError.
If you need to do this kind of thing in general, it's a good idea to use six rather than coding all these things by hand.
Related
I have been writing a script which will recover for me CVSS3 scores when i enter a vulnerability name, i've pretty much got it working as intended except for a minor annoying detail.
π ~/Documents/Tools/Scripts ❯ python3 CVSS3-Grabber.py
Paste Vulnerability Name: PHP 7.2.x < 7.2.21 Multiple Vulnerabilities.
Base Score: None
Vector: <re.Match object; span=(27869, 27913), match='CVSS:3.0/AV:N/AC:L/PR:N/UI:R/S:U/C:L/I:N/A:H'>
Temporal Vector: <re.Match object; span=(27986, 28008), match='CVSS:3.0/E:U/RL:O/RC:C'>
As can be seen the output could be much neater, i would much prefer something like this:
π ~/Documents/Tools/Scripts ❯ python3 CVSS3-Grabber.py
Paste Vulnerability Name: PHP 7.2.x < 7.2.21 Multiple Vulnerabilities.
Base Score: None
Vector: CVSS:3.0/AV:N/AC:L/PR:N/UI:R/S:U/C:L/I:N/A:H
However i have been struggling to figure out how to get the output nicer, is there an easy part of the re module that im missing that can do this for me? or perhaps putting the output into a file first would then allow me to manipulate the text to how i need it.
Here is my code, would appreciate any feedback on how to improve as i have recently gotten back into python and scripting in general.
import requests
import re
from bs4 import BeautifulSoup
from googlesearch import search
def get_url():
vuln = input("Paste Vulnerability Name: ") + "tenable"
for url in search(vuln, tld='com',lang='en',num=1,start=0,stop=1,pause=2.0):
return url
def get_scores(url):
response = requests.get(url)
html = response.text
cvss3_temporal_v = re.search("CVSS:3.0/E:./RL:./RC:.",html)
cvss3_v = re.search("CVSS:3.0/AV:./AC:./PR:./UI:./S:./C:./I:./A:.",html)
cvss3_basescore = re.search("Base Score:....",html)
print("Base Score: ",cvss3_basescore)
print("Vector: ",cvss3_v)
print("Temporal Vector: ",cvss3_temporal_v)
urll = get_url()
get_scores(urll)
### IMPROVEMENTS ###
# Include the base score in output
# Tidy up output
# Vulnerability list?
# modify to accept flags, i.e python3 CVSS3-Grabber.py -v VULNAME ???
# State whether it is a failing issue or Action point
Thanks!
Don't print the match object. Print the match value.
In Python the value is accessible through the .group() method. If there are no regex subgroups (or you want the entire match, like in this case), don't specify any arguments when you call it:
print("Vector: ", cvss3_v.group())
I have two files. An 'initialization' script aka file1.py and a 'main code' script aka 'main_code.py'. The main_code.py is really a several hundred line .ipynb that was converted to a .py file. I want to run the same skeleton of the code with the only adjustment being to pass in the different parameters found in the 'file1.py' script.
In reality, it is much more complex than what I have laid out below with more references to other locations / DBs and what not.
However, I receive errors such as 'each_item[0]' is not defined. I can't seem to be able to pass in the values/variables that come from my loop in file1.py to my script that is contained inside the loop.
Must be doing something very obviously wrong as I imagine this is a simple fix
file1.py:
import pandas as pd
import os
import bumpy as np
import jaydebeapi as jd
#etc...
cities = ['NYC','LA','DC','MIA'] # really comes from a query/column
value_list = [5,500,5000,300] # comes from a query/column
zipped = list(zip(cities,value_list)) # make tuples
for each_item in zipped:
os.system('python loop_file.py')
# where I'm getting errors.
main_code.py:
names = each_item[0]
value = each_item[1]
# lots of things going on here in real code but for simplicity...
print value = value * 4
print value
I have a list including 4000 elements in python which each of its elements is an object of following class with several values.
class Point:
def __init__(self):
self.coords = []
self.IP=[]
self.BW=20
self.status='M'
def __repr__(self):
return str(self.coords)
I do not know how to save this list for future uses.
I have tried to save it by open a file and write() function, but this is not what I want.
I want to save it and import it in next program, like what we do in MATLAB that we can save a variable and import it in future
pickle is a good choice:
import pickle
with open("output.bin", "wb") as output:
pickle.dump(yourList, output)
and symmetric:
import pickle
with open("output.bin", "rb") as data:
yourList = pickle.load(data)
It is a good choice because it is included with the standard library, it can serialize almost any Python object without effort and has a good implementation, although the output is not human readable. Please note that you should use pickle only for your personal scripts, since it will happily load anything it receives, including malicious code: I would not recommend it for production or released projects.
This might be an option:
f = open('foo', 'wb')
np.save(f, my_list)
for loading then use
data = np.load(open('foo'))
if 'b' is not present in 'wb' then the program gives an error:
TypeError: write() argument must be str, not bytes
"b" for binary makes the difference.
Since you say Matlab, numpy should be an option.
f = open('foo', 'w')
np.save(f, my_list)
# later
data = np.load(open('foo'))
Of course, it'll return an array, not a list, but you can coerce it if you really want an array...
I have created a python script that runs from an ArcMap 10.1 session; however, I would like to modify it to run as a stand alone script, if possible. The problem is I don't see a workaround for prompting the user for the parameters when executed outside ArcMap.
Can this even be reasonably done? If so, how would I approach it? Below is a sample of my script. How can I modify this to prompt the user at the command line for the path names of parameters 0 and 1?
import arcpy
arcpy.env.overwriteOutput = True
siteArea = arcpy.GetParameterAsText(0)
tempGDB_Dir = arcpy.GetParameterAsText(1)
tempGDB = tempGDB_Dir + "\\tempGDB.gdb"
# Data from which records will be extracted
redWoods = "D:\\Data\\GIS\\Landforms\\Tress.gdb\\Redwoods"
# List of tree names that will be used in join
treesOfInterest = "C:\\Data\\GIS\\Trees\\RedwoodList.dbf"
inFeature = [redWoods, siteArea]
tempFC = tempGDB_Dir + "\\TempFC"
tempFC_Layer = "TempFC_Layer"
output_dbf = tempGDB_Dir + "\\Output.dbf"
# Make a temporaty geodatabase
arcpy.CreateFileGDB_management(tempGDB_Dir, "tempGDB.gdb")
# Intersect trees with site area
arcpy.Intersect_analysis([redWoods, siteArea], tempFC, "ALL", "", "INPUT")
# Make a temporary feature layer of the results
arcpy.MakeFeatureLayer_management(tempFC, tempFC_Layer)
# Join redwoods data layer to list of trees
arcpy.AddJoin_management(tempFC_Layer, "TreeID", treesOfInterest, "TreeID", "KEEP_COMMON")
# Frequency analysis - keeps only distinct species values
arcpy.Frequency_analysis(tempFC_Layer, output_dbf, "tempFC.TreeID;tempFC.TreeID", "")
# Delete temporary files
arcpy.Delete_management(tempFC_Layer)
arcpy.Delete_management(tempGDB)
This is as much a philosophical question as it is a programmatic one. I am interested in whether this can be done and the amount of effort to do it this way. Is the effort worth the convenience of not opening up a map document?
Check to see if the parameters were specified. If they were not specified, do one of the following:
Use Python's raw_input() method to prompt the user (see this question).
Print a "usage" message, which instructs the user to enter parameters on the command line, and then exit.
Prompting the user could look like this:
siteArea = arcpy.GetParameterAsText(0)
tempGDB_Dir = arcpy.GetParameterAsText(1)
if (not siteArea):
arcpy.AddMessage("Enter the site area:")
siteArea = raw_input()
if (not tempGDB_Dir):
arcpy.AddMessage("Enter the temp GDB dir:")
tempGDB_Dir = raw_input()
Printing a usage message could look like this:
siteArea = arcpy.GetParameterAsText(0)
tempGDB_Dir = arcpy.GetParameterAsText(1)
if (not (siteArea and tempGDB_Dir)):
arcpy.AddMessage("Usage: myscript.py <site area> <temp GDB dir>")
else:
# the rest of your script goes here
If you prompt for input with raw_input(), make sure to make all parameters required when adding to your toolbox in ArcGIS for Desktop. Otherwise, you'll get this error from raw_input() when running in Desktop:
EOFError: EOF when reading a line
hell yeah its worth the convenience of not opening up arcmap. I like to use the optparse module to create command line tools. arcpy.GetParameter(0) is only useful for Esri GUI integration (e.g. script tools). Here is a nice example of a python commandline tool:
http://www.jperla.com/blog/post/a-clean-python-shell-script
I include a unittest class in my tools to testing and automation. I also keep all arcpy.GetParameterAsText statements outside of any real business logic. I like to include at the bottom:
if __name__ == '__main__':
if arcpy.GetParameterAsText(0):
params = parse_arcpy_parameters()
main_business_logic(params)
else:
unittest.main()
What is the design pattern behind python like interactive shell. I want to do this for my server but I am ending up with lot of if - then- else pattern.
For example, when I start python interpreter I get something like this
Python 2.6.7 (r267:88850, Feb 2 2012, 23:50:20)
[GCC 4.5.3] on cygwin
Type "help", "copyright", "credits" or "license" for more information.
>>> help
After help the prompt changes to help
Welcome to Python 2.6! This is the online help utility.
If this is your first time using Python, you should definitely check out
the tutorial on the Internet at http://docs.python.org/tutorial/.
Enter the name of any module, keyword, or topic to get help on writing
Python programs and using Python modules. To quit this help utility and
return to the interpreter, just type "quit".
To get a list of available modules, keywords, or topics, type "modules",
"keywords", or "topics". Each module also comes with a one-line summary
of what it does; to list the modules whose summaries contain a given word
such as "spam", type "modules spam".
help>
I think this is some king of read-eval loop design.
for a REPL, you need a context (an object which stores the REPL's state), a command parser (which parses input and produces an AST), and a way to map commands to actions (actions are generally just functions that modifies the context and/or produces side effects).
A simple REPL can be implemented like the following, where context is implemented using a simple dictionary, AST is just the inputted commands split on whitespaces, and a dictionary is used to map commands to actions:
context = {}
commands = {}
def register(func):
""" convenience function to put `func` into commands map """
# in C++, you cannot introspect the function's name so you would
# need to map the function name to function pointers manually
commands[func.__name__] = func
def parse(s):
""" given a command string `s` produce an AST """
# the simplest parser is just splitting the input string,
# but you can also produce use a more complicated grammer
# to produce a more complicated syntax tree
return s.split()
def do(cmd, commands, context):
""" evaluate the AST, producing an output and/or side effect """
# here, we simply use the first item in the list to choose which function to call
# in more complicated ASTs, the type of the root node can be used to pick actions
return commands[cmd[0]](context, cmd)
#register
def assign(ctx, args):
ctx[args[1]] = args[2]
return '%s = %s' % (args[1], args[2])
#register
def printvar(ctx, args):
print ctx[args[1]]
return None
#register
def defun(ctx, args):
body = ' '.join(args[2:])
ctx[args[1]] = compile(body, '', 'exec')
return 'def %s(): %s' % (args[1], body)
#register
def call(ctx, args):
exec ctx[args[1]] in ctx
return None
# more commands here
context['PS1'] = "> "
while True:
# READ
inp = raw_input(context["PS1"])
# EVAL
cmd = parse(inp)
out = do(cmd, commands, context)
# PRINT
if out is not None: print out
# LOOP
A sample session:
> assign d hello
d = hello
> printvar d
hello
> assign PS1 $
PS1 = $
$defun fun print d + 'world'
def fun(): print d + 'world'
$call fun
helloworld
with a little bit more trickery, you could even merge the context and commands dictionary together, allowing the shell's set of commands to be modified in the shell's language.
The name of this design pattern, if it has a name, is Read-Eval-Print Loop design pattern; so yeah, your question sorta answers itself.