Suppose I have a file called main_config.py and custom_config.py. They are both in the same module.
What I want to do is have two separate config files, where one of them (main) is version controlled, and the other one (custom) is .gitignore'd to allow different setups have their own configs with the main file used as a template.
The code belows used to work properly on Python 3.5, but I was forced to revert to 2.7 and it is no longer working as expected. What am I doing wrong? I'm getting KeyError: 'client2' exception on the line with exec.
Contents of main_config.py:
class MainConfig(object):
clients = {
"client1" = {
"IP" = "127.0.0.1",
"User" = "admin"
}
}
try:
with(open(__file__.replace("main_config.py", "custom_config.py")) as source_file:
exec(source_file.read())
except IOError, e:
print("No custom config detected")
Contents of custom_config.py:
from (...).main_config import MainConfig
MainConfig.clients["client2"]["IP"] = "0.0.0.0"
MainConfig.clients["client2"]["User"] = "root"
I see you use dictionary and = sign instead of :.
Aside from that, you cannot assign IP without client2 assigned before it so your code could look like this:
main_config.py
class MainConfig(object):
clients = {
"client1" : {
"IP" : "127.0.0.1",
"User" : "admin"
}
}
try:
with(open(__file__.replace("main_config.py", "custom_config.py")) as source_file:
exec(source_file.read())
except IOError, e:
print("No custom config detected")
custom_config.py
from (...).main_config import MainConfig
if 'client2' not in MainConfig.clients.keys():
MainConfig.clients["client2"] = {}
MainConfig.clients["client2"]["IP"] = "0.0.0.0"
MainConfig.clients["client2"]["User"] = "root"
I'm not explaining how to solve the specific error you encounter (other answers already do that), but I'd like
to point out there are more suitable config languages for your use case. Specifically, Figura
natively supports this sort of overrideability.
Here's an example:
# main_config.py
class MainConfig:
class clients:
class client1:
IP = "127.0.0.1"
User = "admin"
# custom_config.py
class CustomConfig:
# declare this config is meant to be applied as an override to another
__override__ = True
class client2:
IP = "0.0.0.0"
User = "root"
And combining them in your code:
from figura import build_config
full_config = build_config('main_config.MainConfig', 'custom_config.CustomConfig')
The flexible build_config function treats the first arg as the base config, and the rest as overrides to apply to it.
Full disclosure: I am the author of figura.
Related
I really enjoy the filepond library and would like to implement it in my flask app. Since I was not able to find any useful examples online, I started to write my own, small, proof of concept web application. I would like to upload multiple images to the server and save the filenames in the database. Furthermore, I would like to edit an entry and add additional files or remove the existing ones.
So far I figured out how to upload and revert files before the form is submitted. I am also able to load existing files inside the edit form. Just when I click the 'x' button on a loaded image inside the edit form the image is removed from the filepond window and a 'removefile' event is fired, but the file still remains on the server. Is it possible to trigger the revert request on a loaded file or is there a better solution altogether?
x-button does not remove the file from the server
Here are the relevant snippets from my js file:
FilePond.registerPlugin(
FilePondPluginFileValidateSize,
FilePondPluginImagePreview,
FilePondPluginFileRename,
FilePondPluginFileValidateType
);
inputElement = document.querySelector(".filepond");
token = document
.querySelector('input[name="csrf_token"]')
.getAttribute("value");
FilePond.setOptions({
server: {
headers: { "X-CSRF-TOKEN": token },
process: "./process",
revert: "./revert",
load: {
url: "../",
}
},
});
const filepond = FilePond.create(inputElement, {
// Here I pass the files to my edit form in the following format:
//
// files: [
// {
// source: 'static/images/some_name.png',
// options: {
// type: 'local'
// }
// }]
});
The relevant code from .py file:
#app.route("/process", methods=["POST"])
#app.route("/edit/process", methods=["POST"])
def process():
upload_dir = "static/images"
file_names = []
for key in request.files:
file = request.files[key]
picture_fn = file.filename
file_names.append(picture_fn)
picture_path = os.path.join(upload_dir, picture_fn)
try:
file.save(picture_path)
except:
print("save fail: " + picture_path)
return json.dumps({"filename": [f for f in file_names]})
#app.route("/revert", methods=["DELETE"])
#app.route("/edit/revert", methods=["DELETE"])
def revert():
upload_dir = "static/images"
parsed = json.loads(request.data)
picture_fn = parsed["filename"][0]
picture_path = os.path.join(upload_dir, picture_fn)
try:
os.remove(picture_path)
except:
print("delete fail: " + picture_path)
return json.dumps({"filename": picture_fn})
Here is the repository to my full flask-filepond app:
https://github.com/AugeJevgenij/flask_filepond
Please excuse me if the question is unclear, does not make sense or the code is written poorly.
I just started programming a few months ago.
Acording to filepond documentation you can remove a file stored locally on the server like this:
FilePond.setOptions({
server: {
remove: (source, load, error) {
// 'source' is the path of the file and should be sent to a server endpoint via http
// call the load method before ending the function
load()
}
}
})
then on your server where you receive the source (path), use it to delete the file. Keep in mind that this is a risky approach to get your website hacked!
So, I have a base class that I want to extend most (but not all) of my domain classes from. The goal is that I can add the six audit information columns I need to any domain class with a simple extends. For both creation and updates, I want to log the date, user, and program (based on request URI). It's using a Grails service (called CasService) to find the currently logged on user. The CasService then uses Spring Security and a database call to get the relevant user information for that field.
The trouble is, if I do this, then I'm going to have to Mock the CasService and request object in any unit test that tests a domain that uses these classes. That will also impact unit tests for services and controllers that use these domains. That's going to make unit testing a bit of a pain, and increase boiler plate code, which is what I was trying to avoid.
I'm fishing for better design options, and I'm open to suggestion. My current default is to simply add the same boiler plate to all my domain classes and be done with it. See below for source code and what I've tried so far.
Source
Common Audit Domain Class
package com.mine.common
import grails.util.Holders
import org.springframework.web.context.request.RequestContextHolder
class AuditDomain implements GroovyInterceptable {
def casService = Holders.grailsApplication.mainContext.getBean('casService')
def request = RequestContextHolder?.getRequestAttributes()?.getRequest()
String creator
String creatorProgram
String lastUpdater
String lastUpdateProgram
Date dateCreated
Date lastUpdated
def beforeValidate() {
beforeInsert()
beforeUpdate()
}
def beforeInsert() {
if (this.creator == null) {
this.creator = casService?.getUser() ?: 'unknown'
}
if (this.creatorProgram == null) {
this.creatorProgram = request?.requestURI ?: 'unknown'
}
}
def beforeUpdate() {
this.lastUpdater = casService?.getUser() ?: 'unknown'
this.lastUpdateProgram = request?.requestURI ?: 'unknown'
}
static constraints = {
creator nullable:true, blank: true
lastUpdater nullable:true, blank: true
creatorProgram nullable:true, blank: true
lastUpdateProgram nullable:true, blank: true
}
}
CasService
package com.mine.common
import groovy.sql.Sql
class CasService {
def springSecurityService, sqlService, personService
def getUser() {
if (isLoggedIn()) {
def loginId = springSecurityService.authentication.name.toLowerCase()
def query = "select USER_UNIQUE_ID from some_table where USER_LOGIN = ?"
def parameters = [loginId]
return sqlService.call(query, parameters)
} else {
return null
}
}
def private isLoggedIn() {
if (springSecurityService.isLoggedIn()) {
return true
} else {
log.info "User is not logged in"
return false
}
}
//...
}
What I've Tried
Creating a Test Utilities Class to do the setup logic
I've tried building a class like this:
class AuditTestUtils {
def setup() {
println "Tell AuditDomain to sit down and shut up"
AuditDomain.metaClass.casService = null
AuditDomain.metaClass.request = null
AuditDomain.metaClass.beforeInsert = {}
AuditDomain.metaClass.beforeUpdate = {}
}
def manipulateClass(classToTest) {
classToTest.metaClass.beforeInsert = {println "Yo mama"}
classToTest.metaClass.beforeUpdate = {println "Yo mamak"}
}
}
And then calling it in my Unit Test's setup() and setupSpec() blocks:
def setupSpec() {
def au = new AuditTestUtils()
au.setup()
}
OR
def setupSpec() {
def au = new AuditTestUtils()
au.manipulateClass(TheDomainIAmTesting)
}
No dice. That errors out with a NullPointerException on the CasService as soon as I try to save the domain class that extends the AuditDomain.
java.lang.NullPointerException: Cannot invoke method isLoggedIn() on null object
at com.mine.common.CasService.isLoggedIn(CasService.groovy:127)
at com.mine.common.CasService.getPidm(CasService.groovy:9)
at com.mine.common.AuditDomain.beforeInsert(AuditDomain.groovy:26)
//...
at org.grails.datastore.gorm.GormInstanceApi.save(GormInstanceApi.groovy:161)
at com.mine.common.SomeDomainSpec.test creation(SomeDomainSpec:30)
I'm open to alternate ways of approaching the issue of DRYing the Audit Information out my Domains. I'd settled on inheritance, but there are other ways. Traits aren't available in my environment (Grails 2.3.6), but maybe that's just a reason to get cracking on updating to the latest version.
I'm also open to suggestions about how to test these domains differently. Maybe I should have to contend with the audit columns in the unit tests of every domain class that has them, though I'd rather not. I'm okay with dealing with that in integration tests, but I can unit test the AuditDomain class easily enough on its own. I'd prefer that unit tests on my domains tested the specific things those domains bring to the table, not the common stuff that they all have.
So, ultimately, I've gone with Groovy delegation to meet this need.
The validation logic all lives in
class AuditDomainValidator extends AuditDomainProperties {
def domainClassInstance
public AuditDomainValidator(dci) {
domainClassInstance = dci
}
def beforeValidate() {
def user = defaultUser()
def program = defaultProgram()
if (this.creator == null) {
this.creator = user
}
if (this.creatorProgram == null) {
this.creatorProgram = program
}
this.lastUpdater = user
this.lastUpdateProgram = program
}
private def defaultProgram() {
domainClassInstance.getClass().getCanonicalName()
}
private def defaultUser() {
domainClassInstance.casService?.getUser() ?: 'unknown'
}
}
I created this abstract class to hold the properties while trying various solutions. It could probably be folded into the validator class with no problems, but I'm just lazy enough to leave it in there since it's working.
abstract class AuditDomainProperties {
String creator
String creatorProgram
String lastUpdater
String lastUpdateProgram
Date dateCreated
Date lastUpdated
}
And finally, here's how to implement the validator class in a Grails domain class.
import my.company.CasService
import my.company.AuditDomainValidator
class MyClass {
def casService
#Delegate AuditDomainValidator adv = new AuditDomainValidator(this)
static transients = ['adv']
//...domain class code
static mapping = {
//..domain column mapping
creator column: 'CREATOR'
lastUpdater column: 'LAST_UPADTER'
creatorProgram column: 'CREATOR_PGM'
lastUpdateProgram column: 'LAST_UPDATE_PGM'
dateCreated column: 'DATE_CREATED'
lastUpdated column: 'LAST_UPDATED'
}
}
This approach doesn't work perfectly for Unit and Integration tests, it seems. Trying to access the dateCreated column in either fails with an error that there is no such property for the domain class in question. Thats odd, since running the application works fine. With the Unit tests, I would think it was a mocking issue, but I wouldn't expect that to be a problem in the integration tests.
I'm trying to utilize the build-test-data plugin in my Grails (v2.4.3) app to assist with test data creation for unit testing, but while running my unit tests the plugin cannot find TestDataConfig.groovy to load my specified values (for unique constraint tests, etc).
I've installed the plugin via including it in BuildConfig.groovy:
plugins {
...
test ":build-test-data:2.2.0"
...
}
I've ran the following command to create the TestDataConfig.groovy template, which places the file at \grails-app\conf\:
grails install-build-test-data-config-template
I've followed the general instructions on the plugin wiki to come up with a properly formatted file:
testDataConfig {
sampleData {
'<path>.User' {
def a = 1
username = { -> "username${a++}" }
}
}
}
(Where path is the fully-qualified class name.)
In my tests, I am using the following general format:
import grails.buildtestdata.TestDataConfigurationHolder
import grails.buildtestdata.mixin.Build
import grails.test.mixin.TestFor
import spock.lang.Specification
import spock.lang.Unroll
#TestFor(User)
#Build(User)
class UserSpec extends Specification {
def setup() {
mockForConstraintsTests(User)
TestDataConfigurationHolder.reset()
user = User.buildWithoutSave()
}
#Unroll
void "test #field must be unique"() {
given: 'a User exists'
user.save(flush: true)
when: 'another User is created with a non-unique field value'
def nonUniqueUser = User.buildWithoutSave()
nonUniqueUser."$field" = user."$field"
then: 'validation fails and the field has a unique constraint error'
!nonUniqueUser.validate()
nonUniqueUser.errors.errorCount == 1
nonUniqueUser.errors.getFieldError("$field").code == 'unique'
where:
field << ['username', '<some other field>']
}
}
But, when the test is run (using IntelliJ IDEA) TestDataConfig.groovy cannot be found via the following method in the plugin:
static Class getDefaultTestDataConfigClass() {
GroovyClassLoader classLoader = new GroovyClassLoader(TestDataConfigurationHolder.classLoader)
String testDataConfig = Holders.config?.grails?.buildtestdata?.testDataConfig ?: 'TestDataConfig'
try {
return classLoader.loadClass(testDataConfig)
} catch (ClassNotFoundException ignored) {
log.warn "${testDataConfig}.groovy not found, build-test-data plugin proceeding without config file"
return null
}
}
So the test continues on without a config file and I do not get uniquely generated data sets.
I've even tried explicitly including the file in Config.groovy:
grails.buildtestdata.testDataConfig = "TestDataConfig"
But, the same method in the plugin shows that Holders.config? is null.
I've looked at a few solutions to a similar problem here on StackOverflow with nothing working in my case; I cannot figure out how to get my app to detect the presence of the TestDataConfig.groovy file.
Any ideas? Thanks so much!
Grails 2.2.0
How do I access the custom configuration variables in a Grails domain object constraints.
I would like to have something like this:
class User {
def grailsApplication
String name
static constraints = {
name size: grailsApplication.config.maxlength
}
}
But it fails with "No such property: grailsApplication". I have tried to get it work by following suggestions in getting grails 2.0.0M1 config info in domain object, and static scope? but have not managed to get any combination to work.
How do I access config in domain object constraints? In addition how do I handle such a case in a unit test for the domain constraints?
You can use the grails.util.Holders class to get access to the configuration object as follows:
In Config.groovy:
myMaxSize = 10
In your domain class:
class User {
String name
static constraints = {
name minSize: Holders.config.myMaxSize
}
}
I am trying to access dropbox account info using oauth2 library.
I have got hold of access token from dropbox. Then what I am doing is:
parameters = {
'oauth_consumer_key' : DropboxConstants.app_key, #my app key
'oauth_token' : access_token_g,#token that i got [<key>,<secret>]
'oauth_signature_method': oauth.SignatureMethod_HMAC_SHA1.name,
'oauth_timestamp' : oauth.generate_timestamp(),
'oauth_nonce' : oauth.generate_nonce(),
'oauth_version' : DropboxConstants.api_version,
'oauth_signature' : ''
}
#prepare signature
oauth_request= oauth.Request(method="GET",url=DropboxConstants.account_info_url,parameters=parameters)
signature_method_m = oauth.SignatureMethod_HMAC_SHA1()
signature = signature_method_m.signing_base(consumer=consumer,request=oauth_request,token=access_token_g)
parameters['oauth_signature'] = signature[1]
#prepare url for accessing account info
url = "%s?oauth_token=%s&oauth_consumer_key=%s&oauth_signature_method=%s&oauth_timestamp=%s&oauth_nonce=%s&oauth_version=%s&oauth_signature=%s"%\
(DropboxConstants.account_info_url,access_token['oauth_token'],parameters['oauth_consumer_key'],parameters['oauth_signature_method'],parameters['oauth_timestamp'],parameters['oauth_nonce'],parameters['oauth_version'], parameters['oauth_signature'])
return HttpResponseRedirect(url)
now the signature that is getting generated:
GET&https%3A%2F%2Fapi.dropbox.com%2F0%2Faccount%2Finfo&oauth_consumer_key%3Dedw6k7d78hu8q8v%26oauth_nonce%3D39519001%26oauth_signature_method%3DHMAC-SHA1%26oauth_timestamp%3D1314679561%26oauth_token%3Doauth_token_secret%253Dun58fgoc14n9jlv%2526oauth_token%253D2ew2dafg0r40uwq%26oauth_version%3D1.0
error I get is:
{"error": "Invalid signature. Expected signature base string: GET&https%3A%2F%2Fapi.dropbox.com%2F0%2Faccount%2Finfo&https%253A%252F%252Fapi.dropbox.com%252F0%252Faccount%252Finfo%3D%26oauth_consumer_key%3Dedw6k7d78hu8q8v%26oauth_consumer_key%253Dedw6k7d78hu8q8v%2526oauth_nonce%253D39519001%2526oauth_signature_method%253DHMAC-SHA1%2526oauth_timestamp%253D1314679561%2526oauth_token%253Doauth_token_secret%25253Dun58fgoc14n9jlv%252526oauth_token%25253D2ew2dafg0r40uwq%2526oauth_version%253D1.0%3D%26oauth_nonce%3D39519001%26oauth_signature_method%3DHMAC-SHA1%26oauth_timestamp%3D1314679561%26oauth_token%3D2ew2dafg0r40uwq%26oauth_version%3D1.0"}
firstly please use urlencode to escape chars correctly:
from urllib import urlencode
...
parameters['oauth_token'] = access_token_g['oauth_token']
url = "?".join(DropboxConstants.account_info_url, urlencode(parameters))
see if this helps, if not i'll look into the signiature base
Actually i have solved this problem with a bit change in code as:
access_token_g =
oauth.Token(key=access_token['oauth_token'],secret=access_token['oauth_token_secret'])
#prepare signature
oauth_request = oauth.Request(method="GET",url=account_info_url,parameters=parameters)
signature_method_m = oauth.SignatureMethod_HMAC_SHA1()
oauth_request.sign_request(signature_method=signature_method_m,consumer=consumer,token=access_token_g)
resp, content = client.request(oauth_request.to_url())
It gives me correct content..